Result | FAILURE |
Tests | 45 failed / 721 succeeded |
Started | |
Elapsed | 52m39s |
Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 27 23:25:57.154: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003e6ee60>: { s: "pod \"pod-a1b73ecc-3910-449d-9893-d25c596c3096\" is not Running: timed out waiting for the condition", } pod "pod-a1b73ecc-3910-449d-9893-d25c596c3096" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_02.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:20:55.424: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 27 23:20:56.142: INFO: Creating resource for dynamic PV Nov 27 23:20:56.142: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-8635-e2e-sc6nz96 �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 27 23:25:57.154: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003e6ee60>: { s: "pod \"pod-a1b73ecc-3910-449d-9893-d25c596c3096\" is not Running: timed out waiting for the condition", } pod "pod-a1b73ecc-3910-449d-9893-d25c596c3096" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000d58300) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000d58300) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000d58300, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 27 23:25:57.155: INFO: Deleting pod "pod-a1b73ecc-3910-449d-9893-d25c596c3096" in namespace "volume-expand-8635" Nov 27 23:25:57.305: INFO: Wait up to 5m0s for pod "pod-a1b73ecc-3910-449d-9893-d25c596c3096" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 27 23:25:57.449: INFO: Deleting pod "pod-a1b73ecc-3910-449d-9893-d25c596c3096" in namespace "volume-expand-8635" �[1mSTEP�[0m: Deleting pvc Nov 27 23:25:57.738: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comjvb7z" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-8635". �[1mSTEP�[0m: Found 6 events. Nov 27 23:25:58.193: INFO: At 2022-11-27 23:20:56 +0000 UTC - event for ebs.csi.aws.comjvb7z: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:25:58.193: INFO: At 2022-11-27 23:20:56 +0000 UTC - event for ebs.csi.aws.comjvb7z: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "volume-expand-8635/ebs.csi.aws.comjvb7z" Nov 27 23:25:58.193: INFO: At 2022-11-27 23:20:56 +0000 UTC - event for ebs.csi.aws.comjvb7z: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:25:58.193: INFO: At 2022-11-27 23:21:06 +0000 UTC - event for ebs.csi.aws.comjvb7z: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8635-e2e-sc6nz96": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:25:58.193: INFO: At 2022-11-27 23:24:23 +0000 UTC - event for ebs.csi.aws.comjvb7z: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8635-e2e-sc6nz96": rpc error: code = Internal desc = Could not create volume "pvc-c30270f8-947f-481f-8eaa-1c64b68a46a2": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:25:58.193: INFO: At 2022-11-27 23:25:57 +0000 UTC - event for pod-a1b73ecc-3910-449d-9893-d25c596c3096: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "pod-a1b73ecc-3910-449d-9893-d25c596c3096" not found Nov 27 23:25:58.337: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:25:58.337: INFO: Nov 27 23:25:58.482: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:25:58.626: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 11853 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:23:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:25:27 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:25:58.627: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:25:58.773: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:25:58.944: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:25:58.944: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:25:58.944: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:25:58.944: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:25:58.944: INFO: agnhost-primary-2xz9b started at 2022-11-27 23:25:53 +0000 UTC (0+1 container statuses recorded) Nov 27 23:25:58.944: INFO: Container agnhost-primary ready: true, restart count 0 Nov 27 23:25:58.944: INFO: pod-subpath-test-configmap-pxgt started at 2022-11-27 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 27 23:25:58.944: INFO: Container test-container-subpath-configmap-pxgt ready: true, restart count 0 Nov 27 23:25:58.944: INFO: httpd started at 2022-11-27 23:25:51 +0000 UTC (0+1 container statuses recorded) Nov 27 23:25:58.944: INFO: Container httpd ready: false, restart count 0 Nov 27 23:25:58.944: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:25:58.944: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:25:58.944: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:25:59.474: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:25:59.474: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:25:59.619: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 11027 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1802":"ip-172-20-34-252.sa-east-1.compute.internal","ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:24:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:25:59.619: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:25:59.766: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:25:59.929: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:25:59.929: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:25:59.929: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:25:59.929: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:25:59.929: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:25:59.929: INFO: Container coredns ready: true, restart count 0 Nov 27 23:25:59.929: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:24:42 +0000 UTC (0+7 container statuses recorded) Nov 27 23:25:59.929: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:25:59.929: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:25:59.929: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:25:59.929: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:00.444: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:26:00.444: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:00.589: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 11976 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:23:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:25:28 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:26:00.589: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:00.735: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:00.893: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:00.893: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:00.893: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:00.893: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:00.893: INFO: Container coredns ready: true, restart count 0 Nov 27 23:26:00.893: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:00.893: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:00.893: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:00.893: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:01.393: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:01.393: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:01.538: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 6150 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:26:01.539: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:01.686: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:01.991: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.991: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:26:01.991: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:26:01.991: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:26:01.991: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:26:01.991: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.991: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:26:01.991: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:26:01.991: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:26:01.991: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:26:01.991: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:26:01.991: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:01.991: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:01.991: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.991: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:26:01.991: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.991: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:26:01.992: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.992: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:26:01.992: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:01.992: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:01.992: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:01.992: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:01.992: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:01.992: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:01.992: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:01.992: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.992: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:26:01.992: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:01.992: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:26:02.518: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:02.518: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:02.663: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 13972 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1699":"ip-172-20-58-210.sa-east-1.compute.internal","ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:25:49 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:25:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:56 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:56 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:25:56 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:25:56 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-1699^d2d122fe-6eaa-11ed-92df-7228b7de345b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-1699^d2d122fe-6eaa-11ed-92df-7228b7de345b,DevicePath:,},},Config:nil,},} Nov 27 23:26:02.664: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:02.811: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:02.974: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:02.974: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:02.974: INFO: pod-subpath-test-dynamicpv-dfp2 started at 2022-11-27 23:25:48 +0000 UTC (2+1 container statuses recorded) Nov 27 23:26:02.974: INFO: Init container init-volume-dynamicpv-dfp2 ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Init container test-init-subpath-dynamicpv-dfp2 ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container test-container-subpath-dynamicpv-dfp2 ready: false, restart count 0 Nov 27 23:26:02.974: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:02.974: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:02.974: INFO: test-webserver-48243c59-6a81-4fab-8553-ae5f710c7444 started at 2022-11-27 23:25:01 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:02.974: INFO: Container test-webserver ready: true, restart count 0 Nov 27 23:26:02.974: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:25:45 +0000 UTC (0+7 container statuses recorded) Nov 27 23:26:02.974: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:26:02.974: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:02.975: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:02.975: INFO: liveness-33e27314-ba60-45c8-9252-4faa7e734eaf started at 2022-11-27 23:25:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:02.975: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:26:03.478: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:03.478: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-8635" for this suite.
Find pod-a1b73ecc-3910-449d-9893-d25c596c3096 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 27 23:38:53.927: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc00445af90>: { s: "pod \"pod-1f8f41da-9be2-47fd-8df5-3f847db28f40\" is not Running: timed out waiting for the condition", } pod "pod-1f8f41da-9be2-47fd-8df5-3f847db28f40" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_11.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:33:52.177: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 27 23:33:52.900: INFO: Creating resource for dynamic PV Nov 27 23:33:52.901: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-1820-e2e-scnkgp2 �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 27 23:38:53.927: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc00445af90>: { s: "pod \"pod-1f8f41da-9be2-47fd-8df5-3f847db28f40\" is not Running: timed out waiting for the condition", } pod "pod-1f8f41da-9be2-47fd-8df5-3f847db28f40" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000b30180) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000b30180) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000b30180, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 27 23:38:53.928: INFO: Deleting pod "pod-1f8f41da-9be2-47fd-8df5-3f847db28f40" in namespace "volume-expand-1820" Nov 27 23:38:54.077: INFO: Wait up to 5m0s for pod "pod-1f8f41da-9be2-47fd-8df5-3f847db28f40" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 27 23:38:54.222: INFO: Deleting pod "pod-1f8f41da-9be2-47fd-8df5-3f847db28f40" in namespace "volume-expand-1820" �[1mSTEP�[0m: Deleting pvc Nov 27 23:38:54.510: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comkbcfp" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-1820". �[1mSTEP�[0m: Found 6 events. Nov 27 23:38:54.947: INFO: At 2022-11-27 23:33:53 +0000 UTC - event for ebs.csi.aws.comkbcfp: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:38:54.947: INFO: At 2022-11-27 23:33:53 +0000 UTC - event for ebs.csi.aws.comkbcfp: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:38:54.947: INFO: At 2022-11-27 23:33:53 +0000 UTC - event for ebs.csi.aws.comkbcfp: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "volume-expand-1820/ebs.csi.aws.comkbcfp" Nov 27 23:38:54.947: INFO: At 2022-11-27 23:34:03 +0000 UTC - event for ebs.csi.aws.comkbcfp: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-1820-e2e-scnkgp2": rpc error: code = Internal desc = Could not create volume "pvc-c2da5204-1808-472e-b246-239dfabd4d13": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:38:54.947: INFO: At 2022-11-27 23:34:14 +0000 UTC - event for ebs.csi.aws.comkbcfp: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-1820-e2e-scnkgp2": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:38:54.947: INFO: At 2022-11-27 23:38:54 +0000 UTC - event for pod-1f8f41da-9be2-47fd-8df5-3f847db28f40: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "pod-1f8f41da-9be2-47fd-8df5-3f847db28f40" not found Nov 27 23:38:55.092: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:38:55.092: INFO: Nov 27 23:38:55.238: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:38:55.383: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 35059 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:37:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:38:55.383: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:38:55.530: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:38:55.679: INFO: pod-subpath-test-inlinevolume-4f44 started at 2022-11-27 23:38:46 +0000 UTC (2+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Init container init-volume-inlinevolume-4f44 ready: true, restart count 0 Nov 27 23:38:55.679: INFO: Init container test-init-volume-inlinevolume-4f44 ready: true, restart count 0 Nov 27 23:38:55.679: INFO: Container test-container-subpath-inlinevolume-4f44 ready: false, restart count 0 Nov 27 23:38:55.679: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:38:55.679: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:38:55.679: INFO: netserver-0 started at 2022-11-27 23:38:07 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:55.679: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:55.679: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:38:55.679: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:55.679: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:38:55.679: INFO: inline-volume-r55rg started at 2022-11-27 23:37:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Container volume-tester ready: false, restart count 0 Nov 27 23:38:55.679: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-4b77r started at 2022-11-27 23:38:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:38:55.679: INFO: startup-ed1b587a-5f15-4fa7-b254-5826c007451d started at 2022-11-27 23:35:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Container busybox ready: false, restart count 0 Nov 27 23:38:55.679: INFO: netserver-0 started at 2022-11-27 23:38:23 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:55.679: INFO: simpletest.rc-2zw65 started at 2022-11-27 23:38:35 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:55.679: INFO: Container nginx ready: true, restart count 0 Nov 27 23:38:56.192: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:38:56.192: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:38:56.337: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 35019 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:37:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:38:56.338: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:38:56.486: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:38:56.635: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:38:56.635: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:38:56.635: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:38:56.635: INFO: simpletest.rc-pfv8n started at 2022-11-27 23:38:35 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container nginx ready: true, restart count 0 Nov 27 23:38:56.635: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container coredns ready: true, restart count 0 Nov 27 23:38:56.635: INFO: test-container-pod started at 2022-11-27 23:38:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:56.635: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:56.635: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:38:56.635: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:56.635: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:38:56.635: INFO: test-rolling-update-with-lb-7db9cf5ff8-ttckf started at 2022-11-27 23:36:20 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container agnhost ready: true, restart count 0 Nov 27 23:38:56.635: INFO: netserver-1 started at 2022-11-27 23:38:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:56.635: INFO: test-container-pod started at 2022-11-27 23:38:56 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container webserver ready: false, restart count 0 Nov 27 23:38:56.635: INFO: netserver-1 started at 2022-11-27 23:38:07 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:56.635: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:57.222: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:38:57.222: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:38:57.367: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 36479 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-9407":"ip-172-20-50-151.sa-east-1.compute.internal","csi-mock-csi-mock-volumes-887":"csi-mock-csi-mock-volumes-887","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2022-11-27 23:37:58 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:38:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:38:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:38:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:38:28 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd kubernetes.io/csi/csi-mock-csi-mock-volumes-887^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-887^4,DevicePath:,},},Config:nil,},} Nov 27 23:38:57.368: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:38:57.514: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:38:57.672: INFO: csi-mockplugin-resizer-0 started at 2022-11-27 23:37:51 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:38:57.672: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container coredns ready: true, restart count 0 Nov 27 23:38:57.672: INFO: csi-mockplugin-attacher-0 started at 2022-11-27 23:37:51 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:38:57.672: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:57.672: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:38:57.672: INFO: webserver-pod started at 2022-11-27 23:34:14 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container agnhost ready: false, restart count 0 Nov 27 23:38:57.672: INFO: netserver-2 started at 2022-11-27 23:38:07 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:57.672: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:38:37 +0000 UTC (0+7 container statuses recorded) Nov 27 23:38:57.672: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:38:57.672: INFO: pvc-volume-tester-tzggl started at 2022-11-27 23:37:57 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container volume-tester ready: false, restart count 0 Nov 27 23:38:57.672: INFO: netserver-2 started at 2022-11-27 23:38:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:57.672: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:38:57.672: INFO: test-rolling-update-with-lb-7db9cf5ff8-xpcdf started at 2022-11-27 23:36:20 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:57.672: INFO: Container agnhost ready: true, restart count 0 Nov 27 23:38:57.672: INFO: csi-mockplugin-0 started at 2022-11-27 23:37:51 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:57.672: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:38:57.672: INFO: Container mock ready: true, restart count 0 Nov 27 23:38:58.236: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:38:58.236: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:38:58.381: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 34164 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:38:58.382: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:38:58.528: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:38:58.678: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.678: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:38:58.678: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:38:58.679: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:38:58.679: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:38:58.679: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:38:58.679: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:58.679: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:38:58.679: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:38:58.679: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:38:58.679: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:38:58.679: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:58.679: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:58.679: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:38:58.679: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:38:58.679: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:58.679: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:38:59.274: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:38:59.274: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:38:59.419: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 35855 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6021":"csi-mock-csi-mock-volumes-6021","ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:37:38 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:37:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:38:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:38:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:38:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:38:17 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-6021^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-6021^4,DevicePath:,},},Config:nil,},} Nov 27 23:38:59.419: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:38:59.567: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:38:59.716: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:38:59.716: INFO: test-rolling-update-with-lb-7db9cf5ff8-rzt9m started at 2022-11-27 23:36:20 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container agnhost ready: true, restart count 0 Nov 27 23:38:59.716: INFO: netserver-3 started at 2022-11-27 23:38:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container webserver ready: true, restart count 0 Nov 27 23:38:59.716: INFO: pod-subpath-test-preprovisionedpv-tc58 started at 2022-11-27 23:38:55 +0000 UTC (2+2 container statuses recorded) Nov 27 23:38:59.716: INFO: Init container init-volume-preprovisionedpv-tc58 ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Init container test-init-subpath-preprovisionedpv-tc58 ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Container test-container-subpath-preprovisionedpv-tc58 ready: false, restart count 0 Nov 27 23:38:59.716: INFO: Container test-container-volume-preprovisionedpv-tc58 ready: false, restart count 0 Nov 27 23:38:59.716: INFO: csi-mockplugin-attacher-0 started at 2022-11-27 23:37:30 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:38:59.716: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:59.716: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:38:59.716: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-8gr99 started at 2022-11-27 23:38:35 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:38:59.716: INFO: pvc-volume-tester-gwtvt started at 2022-11-27 23:37:37 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container volume-tester ready: true, restart count 0 Nov 27 23:38:59.716: INFO: netserver-3 started at 2022-11-27 23:38:59 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container webserver ready: false, restart count 0 Nov 27 23:38:59.716: INFO: pod-subpath-test-inlinevolume-sspq started at 2022-11-27 23:38:59 +0000 UTC (1+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Init container init-volume-inlinevolume-sspq ready: false, restart count 0 Nov 27 23:38:59.716: INFO: Container test-container-subpath-inlinevolume-sspq ready: false, restart count 0 Nov 27 23:38:59.716: INFO: csi-mockplugin-0 started at 2022-11-27 23:37:30 +0000 UTC (0+3 container statuses recorded) Nov 27 23:38:59.716: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:38:59.716: INFO: Container mock ready: true, restart count 0 Nov 27 23:38:59.716: INFO: netserver-3 started at 2022-11-27 23:38:07 +0000 UTC (0+1 container statuses recorded) Nov 27 23:38:59.716: INFO: Container webserver ready: true, restart count 0 Nov 27 23:39:00.361: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:39:00.361: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-1820" for this suite.
Find pod-1f8f41da-9be2-47fd-8df5-3f847db28f40 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 27 23:42:13.047: Unexpected error: <*errors.errorString | 0xc0002ca280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385from junit_24.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:37:11.306: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volumemode �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should not mount / map unused volumes in a pod [LinuxOnly] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 27 23:37:12.028: INFO: Creating resource for dynamic PV Nov 27 23:37:12.028: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volumemode-3688-e2e-scz6zdw �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating pod Nov 27 23:42:13.046: FAIL: Unexpected error: <*errors.errorString | 0xc0002ca280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385 +0x6c5 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001cd680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0001cd680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0001cd680, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 27 23:42:13.047: INFO: Deleting pod "pod-bfe6761c-23e1-4f62-9192-87a927443e5c" in namespace "volumemode-3688" Nov 27 23:42:13.196: INFO: Wait up to 5m0s for pod "pod-bfe6761c-23e1-4f62-9192-87a927443e5c" to be fully deleted �[1mSTEP�[0m: Deleting pvc Nov 27 23:42:13.491: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com2bbkv" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volumemode-3688". �[1mSTEP�[0m: Found 5 events. Nov 27 23:42:13.940: INFO: At 2022-11-27 23:37:12 +0000 UTC - event for ebs.csi.aws.com2bbkv: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:42:13.940: INFO: At 2022-11-27 23:37:12 +0000 UTC - event for ebs.csi.aws.com2bbkv: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "volumemode-3688/ebs.csi.aws.com2bbkv" Nov 27 23:42:13.940: INFO: At 2022-11-27 23:37:12 +0000 UTC - event for ebs.csi.aws.com2bbkv: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:42:13.940: INFO: At 2022-11-27 23:37:22 +0000 UTC - event for ebs.csi.aws.com2bbkv: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-3688-e2e-scz6zdw": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:42:13.940: INFO: At 2022-11-27 23:42:13 +0000 UTC - event for pod-bfe6761c-23e1-4f62-9192-87a927443e5c: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "pod-bfe6761c-23e1-4f62-9192-87a927443e5c" not found Nov 27 23:42:14.084: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:42:14.084: INFO: Nov 27 23:42:14.230: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:42:14.374: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 42244 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8948":"csi-mock-csi-mock-volumes-8948","ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:41:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:42:14.375: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:42:14.525: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:42:14.673: INFO: simpletest.rc-zm5zh started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:14.674: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:14.674: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:42:14.674: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:42:14.674: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:42:14.674: INFO: deployment-585449566-p56rv started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:14.674: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:14.674: INFO: pod-subpath-test-preprovisionedpv-jm5v started at 2022-11-27 23:42:12 +0000 UTC (1+2 container statuses recorded) Nov 27 23:42:14.674: INFO: Init container test-init-subpath-preprovisionedpv-jm5v ready: false, restart count 0 Nov 27 23:42:14.674: INFO: Container test-container-subpath-preprovisionedpv-jm5v ready: false, restart count 0 Nov 27 23:42:14.674: INFO: Container test-container-volume-preprovisionedpv-jm5v ready: false, restart count 0 Nov 27 23:42:14.674: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:42:14.674: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:42:14.674: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:14.674: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:42:14.674: INFO: csi-mockplugin-0 started at 2022-11-27 23:41:49 +0000 UTC (0+4 container statuses recorded) Nov 27 23:42:14.674: INFO: Container busybox ready: true, restart count 0 Nov 27 23:42:14.674: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:42:14.674: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:42:14.674: INFO: Container mock ready: true, restart count 0 Nov 27 23:42:14.674: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-vb2wd started at 2022-11-27 23:41:52 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:14.674: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:42:14.674: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-k2kjs started at 2022-11-27 23:41:55 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:14.674: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:42:14.674: INFO: simpletest.rc-wsd6h started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:14.674: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:15.374: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:42:15.374: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:42:15.518: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 42301 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-4495":"ip-172-20-34-252.sa-east-1.compute.internal","ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:41:49 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:41:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:41:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:41:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:41:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:41:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-4495^0e489415-6ead-11ed-9785-42915cc19d0d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-4495^0e489415-6ead-11ed-9785-42915cc19d0d,DevicePath:,},},Config:nil,},} Nov 27 23:42:15.519: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:42:15.665: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:42:15.819: INFO: simpletest.rc-jwzjw started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:15.819: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:15.819: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:42:15.819: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:42:15.819: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:41:43 +0000 UTC (0+7 container statuses recorded) Nov 27 23:42:15.819: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:15.819: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:42:15.819: INFO: hostpath-client started at 2022-11-27 23:42:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:15.819: INFO: Container hostpath-client ready: false, restart count 0 Nov 27 23:42:15.819: INFO: affinity-nodeport-transition-bbwqx started at 2022-11-27 23:42:06 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:15.819: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Nov 27 23:42:15.819: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:42:15.820: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:42:15.820: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:42:15.820: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:15.820: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:42:15.820: INFO: simpletest.rc-7nqjt started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:15.820: INFO: Container nginx ready: true, restart count 0 Nov 27 23:42:15.820: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:15.820: INFO: Container coredns ready: true, restart count 0 Nov 27 23:42:16.380: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:42:16.380: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:42:16.532: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 39938 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:42:16.533: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:42:16.679: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:42:16.972: INFO: execpod-affinity49jwg started at <nil> (0+0 container statuses recorded) Nov 27 23:42:16.972: INFO: termination-message-container8c536a81-4ba3-451c-9fdb-17aa89be8650 started at <nil> (0+0 container statuses recorded) Nov 27 23:42:16.972: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:42:16.972: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:42:16.972: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:16.972: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:42:16.972: INFO: pod-terminate-status-1-13 started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container fail ready: false, restart count 0 Nov 27 23:42:16.972: INFO: simpletest.rc-jtqkf started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:16.972: INFO: concurrent-27826542-qf44l started at 2022-11-27 23:42:00 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container c ready: true, restart count 0 Nov 27 23:42:16.972: INFO: security-context-2f02dd67-d20f-4111-942e-4bfbab94922c started at 2022-11-27 23:42:08 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container test-container ready: false, restart count 0 Nov 27 23:42:16.972: INFO: concurrent-27826541-ntmvw started at 2022-11-27 23:41:00 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container c ready: true, restart count 0 Nov 27 23:42:16.972: INFO: simpletest.rc-rgpfg started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:16.972: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:42:16.972: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:42:16.972: INFO: deployment-585449566-42q9m started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:16.972: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container coredns ready: true, restart count 0 Nov 27 23:42:16.972: INFO: affinity-nodeport-transition-fj6rc started at 2022-11-27 23:42:06 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.972: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Nov 27 23:42:16.972: INFO: sample-webhook-deployment-5d5bc8ffd5-4c2k9 started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.973: INFO: Container sample-webhook ready: false, restart count 0 Nov 27 23:42:16.973: INFO: simpletest.rc-xnt96 started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:16.973: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:17.817: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:42:17.817: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:42:17.962: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 34164 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:42:17.962: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:42:18.111: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:42:18.261: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.261: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:42:18.261: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:42:18.261: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:42:18.261: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:42:18.261: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.261: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:42:18.261: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:42:18.261: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:42:18.261: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:42:18.261: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:42:18.261: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:42:18.261: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:18.261: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.261: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:42:18.261: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.261: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:42:18.261: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.261: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:42:18.261: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:42:18.261: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:42:18.261: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:42:18.261: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:42:18.261: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:42:18.262: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:18.262: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:42:18.262: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.262: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:42:18.262: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:18.262: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:42:18.828: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:42:18.828: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:42:18.972: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 39463 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:39:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:42:18.972: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:42:19.119: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:42:19.267: INFO: pod-projected-secrets-aa5f7980-9910-40b2-98ca-050a23a29d42 started at 2022-11-27 23:41:49 +0000 UTC (0+3 container statuses recorded) Nov 27 23:42:19.267: INFO: Container creates-volume-test ready: true, restart count 0 Nov 27 23:42:19.267: INFO: Container dels-volume-test ready: true, restart count 0 Nov 27 23:42:19.267: INFO: Container upds-volume-test ready: true, restart count 0 Nov 27 23:42:19.267: INFO: simpletest.rc-m2bf5 started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container nginx ready: true, restart count 0 Nov 27 23:42:19.267: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:42:19.267: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:42:19.267: INFO: simpletest.rc-7j944 started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container nginx ready: true, restart count 0 Nov 27 23:42:19.267: INFO: deployment-585449566-kkvn4 started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container nginx ready: false, restart count 0 Nov 27 23:42:19.267: INFO: affinity-nodeport-transition-m4qdv started at 2022-11-27 23:42:06 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Nov 27 23:42:19.267: INFO: simpletest.rc-nkxfx started at 2022-11-27 23:42:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container nginx ready: true, restart count 0 Nov 27 23:42:19.267: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:42:19.267: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:42:19.267: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:42:19.267: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:42:19.267: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-tvwql started at 2022-11-27 23:42:02 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:42:19.267: INFO: pod-projected-secrets-f0c90dd7-f6ad-4b66-acdf-83ac6918a00e started at 2022-11-27 23:42:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container projected-secret-volume-test ready: false, restart count 0 Nov 27 23:42:19.267: INFO: pod-4103451a-66f0-4a51-90a0-6ee811b0d41e started at 2022-11-27 23:42:09 +0000 UTC (0+1 container statuses recorded) Nov 27 23:42:19.267: INFO: Container write-pod ready: true, restart count 0 Nov 27 23:42:19.808: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:42:19.808: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volumemode-3688" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumes\sshould\sstore\sdata$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 27 23:35:38.633: Failed to create injector pod: timed out waiting for the condition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186from junit_23.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:30:36.586: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should store data /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 27 23:30:37.314: INFO: Creating resource for dynamic PV Nov 27 23:30:37.314: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volume-5270-e2e-sc65htk �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: starting external-injector �[1mSTEP�[0m: Deleting pod external-injector in namespace volume-5270 Nov 27 23:35:38.488: INFO: Waiting for pod external-injector to disappear Nov 27 23:35:38.632: INFO: Pod external-injector no longer exists Nov 27 23:35:38.633: FAIL: Failed to create injector pod: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumesTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186 +0x3ff k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00025ed80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00025ed80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00025ed80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: cleaning the environment after external �[1mSTEP�[0m: Deleting pvc Nov 27 23:35:38.779: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comcxhnh" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-5270". �[1mSTEP�[0m: Found 7 events. Nov 27 23:35:39.217: INFO: At 2022-11-27 23:30:37 +0000 UTC - event for ebs.csi.aws.comcxhnh: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:35:39.217: INFO: At 2022-11-27 23:30:37 +0000 UTC - event for ebs.csi.aws.comcxhnh: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:35:39.218: INFO: At 2022-11-27 23:30:37 +0000 UTC - event for ebs.csi.aws.comcxhnh: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "volume-5270/ebs.csi.aws.comcxhnh" Nov 27 23:35:39.218: INFO: At 2022-11-27 23:30:47 +0000 UTC - event for ebs.csi.aws.comcxhnh: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-5270-e2e-sc65htk": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:35:39.218: INFO: At 2022-11-27 23:32:06 +0000 UTC - event for ebs.csi.aws.comcxhnh: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-5270-e2e-sc65htk": rpc error: code = Aborted desc = Create volume request for pvc-9c80e17b-71a5-44e1-b956-bcb36f169359 is already in progress Nov 27 23:35:39.218: INFO: At 2022-11-27 23:35:06 +0000 UTC - event for ebs.csi.aws.comcxhnh: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-5270-e2e-sc65htk": rpc error: code = Internal desc = Could not create volume "pvc-9c80e17b-71a5-44e1-b956-bcb36f169359": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:35:39.218: INFO: At 2022-11-27 23:35:38 +0000 UTC - event for external-injector: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "external-injector" not found Nov 27 23:35:39.362: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:35:39.362: INFO: Nov 27 23:35:39.507: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:39.652: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 30106 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:34:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:39.652: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:39.799: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:39.958: INFO: busybox-0a751258-a7fc-4812-9922-7655a15f522e started at 2022-11-27 23:35:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:39.959: INFO: Container busybox ready: true, restart count 0 Nov 27 23:35:39.959: INFO: liveness-13d8a8f9-feb3-4cfb-8d05-b50b8a6dff39 started at 2022-11-27 23:34:21 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:39.959: INFO: Container agnhost-container ready: true, restart count 3 Nov 27 23:35:39.959: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:39.959: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:39.959: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:39.959: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:39.959: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:39.959: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:39.959: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:39.959: INFO: startup-ed1b587a-5f15-4fa7-b254-5826c007451d started at 2022-11-27 23:35:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:39.959: INFO: Container busybox ready: false, restart count 0 Nov 27 23:35:40.473: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:40.473: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:40.618: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 30783 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:30:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:40.618: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:40.765: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:40.913: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:40.913: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:40.913: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:40.913: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:40.913: INFO: pause-pod-1 started at 2022-11-27 23:34:19 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:40.913: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:40.913: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:40.913: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:40.913: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:40.913: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:40.913: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:35:40.913: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:40.913: INFO: Container coredns ready: true, restart count 0 Nov 27 23:35:40.913: INFO: pod-secrets-326bd201-988e-4ecd-9c1b-d3fddb973706 started at 2022-11-27 23:34:30 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:40.913: INFO: Container creates-volume-test ready: true, restart count 0 Nov 27 23:35:40.913: INFO: Container dels-volume-test ready: true, restart count 0 Nov 27 23:35:40.913: INFO: Container upds-volume-test ready: true, restart count 0 Nov 27 23:35:40.913: INFO: pod-init-9687a132-00f3-490b-aa3f-f3bb8d56d65b started at 2022-11-27 23:35:25 +0000 UTC (2+1 container statuses recorded) Nov 27 23:35:40.913: INFO: Init container init1 ready: false, restart count 1 Nov 27 23:35:40.913: INFO: Init container init2 ready: false, restart count 0 Nov 27 23:35:40.913: INFO: Container run1 ready: false, restart count 0 Nov 27 23:35:41.464: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:41.464: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:41.609: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 30281 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:41.609: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:41.759: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:41.909: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:41.909: INFO: Container coredns ready: true, restart count 0 Nov 27 23:35:41.909: INFO: logs-generator started at 2022-11-27 23:35:35 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:41.909: INFO: Container logs-generator ready: true, restart count 0 Nov 27 23:35:41.909: INFO: webserver-pod started at 2022-11-27 23:34:14 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:41.909: INFO: Container agnhost ready: false, restart count 0 Nov 27 23:35:41.909: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:41.909: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:41.909: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:41.909: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:41.909: INFO: hairpin started at 2022-11-27 23:35:35 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:41.909: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:41.909: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:41.909: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:41.909: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:41.909: INFO: pod-exec-websocket-f87c3400-de5c-4ba8-8963-0b8226f6424b started at 2022-11-27 23:35:39 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:41.909: INFO: Container main ready: true, restart count 0 Nov 27 23:35:42.456: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:42.456: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:35:42.601: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 25553 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:42.602: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:35:42.748: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:35:42.902: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:35:42.903: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:35:42.903: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:35:42.903: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:35:42.903: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:35:42.903: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:42.903: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:35:42.903: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:35:42.903: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:35:42.903: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:42.903: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:42.903: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:42.903: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:42.903: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:35:42.903: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:42.903: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:35:43.404: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:35:43.404: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:35:43.549: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 31131 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-4879":"ip-172-20-58-210.sa-east-1.compute.internal","ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:35:28 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:35:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:35:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:35:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:35:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:35:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-4879^2bcd8909-6eac-11ed-aff2-ca17b4810a4c],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-4879^2bcd8909-6eac-11ed-aff2-ca17b4810a4c,DevicePath:,},},Config:nil,},} Nov 27 23:35:43.550: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:35:43.696: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:35:43.852: INFO: pod-projected-configmaps-082b52b2-3797-4652-a44c-7b7cd459989e started at 2022-11-27 23:34:53 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:43.852: INFO: Container createcm-volume-test ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container delcm-volume-test ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container updcm-volume-test ready: true, restart count 0 Nov 27 23:35:43.852: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-xn7pz started at 2022-11-27 23:35:27 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:43.852: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:43.852: INFO: exec-volume-test-preprovisionedpv-rzw7 started at 2022-11-27 23:35:40 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:43.852: INFO: Container exec-container-preprovisionedpv-rzw7 ready: false, restart count 0 Nov 27 23:35:43.852: INFO: pod-ephm-test-projected-btw8 started at 2022-11-27 23:34:15 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:43.852: INFO: Container test-container-subpath-projected-btw8 ready: false, restart count 0 Nov 27 23:35:43.852: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:43.852: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:43.852: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:35:24 +0000 UTC (0+7 container statuses recorded) Nov 27 23:35:43.852: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:43.852: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:43.852: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:43.852: INFO: pod-subpath-test-dynamicpv-v6dw started at 2022-11-27 23:35:27 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:43.852: INFO: Init container init-volume-dynamicpv-v6dw ready: true, restart count 0 Nov 27 23:35:43.852: INFO: Container test-container-subpath-dynamicpv-v6dw ready: true, restart count 0 Nov 27 23:35:45.015: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:35:45.015: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-5270" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 27 23:40:30.348: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc004c68c00>: { s: "pod \"pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2\" is not Running: timed out waiting for the condition", } pod "pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_19.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:35:28.609: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 27 23:35:29.332: INFO: Creating resource for dynamic PV Nov 27 23:35:29.332: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-6961-e2e-sc4vv9c �[1mSTEP�[0m: creating a claim Nov 27 23:35:29.478: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 27 23:40:30.348: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc004c68c00>: { s: "pod \"pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2\" is not Running: timed out waiting for the condition", } pod "pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001a7e00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0001a7e00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0001a7e00, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 27 23:40:30.348: INFO: Deleting pod "pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2" in namespace "volume-expand-6961" Nov 27 23:40:30.497: INFO: Wait up to 5m0s for pod "pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 27 23:40:30.640: INFO: Deleting pod "pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2" in namespace "volume-expand-6961" �[1mSTEP�[0m: Deleting pvc Nov 27 23:40:30.928: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comcw5lx" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-6961". �[1mSTEP�[0m: Found 5 events. Nov 27 23:40:31.366: INFO: At 2022-11-27 23:35:29 +0000 UTC - event for ebs.csi.aws.comcw5lx: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:40:31.366: INFO: At 2022-11-27 23:35:29 +0000 UTC - event for ebs.csi.aws.comcw5lx: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:40:31.366: INFO: At 2022-11-27 23:35:29 +0000 UTC - event for ebs.csi.aws.comcw5lx: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "volume-expand-6961/ebs.csi.aws.comcw5lx" Nov 27 23:40:31.366: INFO: At 2022-11-27 23:35:39 +0000 UTC - event for ebs.csi.aws.comcw5lx: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-6961-e2e-sc4vv9c": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:40:31.366: INFO: At 2022-11-27 23:40:30 +0000 UTC - event for pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2" not found Nov 27 23:40:31.510: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:40:31.510: INFO: Nov 27 23:40:31.655: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:40:31.799: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 38644 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:37:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:40:31.800: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:40:31.947: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:40:32.100: INFO: httpd started at 2022-11-27 23:40:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:32.100: INFO: Container httpd ready: false, restart count 0 Nov 27 23:40:32.100: INFO: ss2-1 started at 2022-11-27 23:40:23 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:32.100: INFO: Container webserver ready: true, restart count 0 Nov 27 23:40:32.100: INFO: busybox-user-65534-206fe4b7-9dcc-4a82-b9f3-f4c03961ea93 started at 2022-11-27 23:40:26 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:32.100: INFO: Container busybox-user-65534-206fe4b7-9dcc-4a82-b9f3-f4c03961ea93 ready: false, restart count 0 Nov 27 23:40:32.100: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:40:32.100: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:40:32.100: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:40:32.100: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:40:32.100: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:40:32.100: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:32.100: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:40:32.100: INFO: startup-01d5e15a-f4f6-430d-8ae7-885912fcf4a4 started at 2022-11-27 23:39:40 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:32.100: INFO: Container busybox ready: true, restart count 0 Nov 27 23:40:32.600: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:40:32.600: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:40:32.745: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 35019 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:37:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:40:32.746: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:40:32.893: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:40:33.042: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:40:33.042: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:40:33.042: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:40:33.042: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:33.042: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:40:33.042: INFO: ss2-0 started at 2022-11-27 23:40:27 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:33.042: INFO: Container webserver ready: true, restart count 0 Nov 27 23:40:33.042: INFO: run-test started at 2022-11-27 23:40:25 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:33.042: INFO: Container run-test ready: false, restart count 0 Nov 27 23:40:33.042: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:33.042: INFO: Container coredns ready: true, restart count 0 Nov 27 23:40:33.042: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:40:33.042: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:40:33.042: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:33.042: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:40:33.042: INFO: test-rolling-update-with-lb-5f85c5c5f4-94k6q started at 2022-11-27 23:39:30 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:33.042: INFO: Container agnhost ready: true, restart count 0 Nov 27 23:40:33.590: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:40:33.590: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:40:33.734: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 38853 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-9941":"ip-172-20-50-151.sa-east-1.compute.internal","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2022-11-27 23:37:58 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:08 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:08 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:08 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:40:08 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd kubernetes.io/csi/csi-hostpath-ephemeral-9941^b4d8b64f-6eac-11ed-8ba3-ba529b20c548],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-9941^b4d8b64f-6eac-11ed-8ba3-ba529b20c548,DevicePath:,},},Config:nil,},} Nov 27 23:40:33.735: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:40:33.882: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:40:34.034: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:34.034: INFO: Container coredns ready: true, restart count 0 Nov 27 23:40:34.034: INFO: hostexec-ip-172-20-50-151.sa-east-1.compute.internal-mprs5 started at 2022-11-27 23:40:22 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:34.034: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:40:34.034: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:40:34.034: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:40:34.034: INFO: pod-subpath-test-inlinevolume-kwcf started at 2022-11-27 23:40:31 +0000 UTC (2+2 container statuses recorded) Nov 27 23:40:34.034: INFO: Init container init-volume-inlinevolume-kwcf ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Init container test-init-subpath-inlinevolume-kwcf ready: false, restart count 0 Nov 27 23:40:34.034: INFO: Container test-container-subpath-inlinevolume-kwcf ready: false, restart count 0 Nov 27 23:40:34.034: INFO: Container test-container-volume-inlinevolume-kwcf ready: false, restart count 0 Nov 27 23:40:34.034: INFO: test-rolling-update-with-lb-5f85c5c5f4-68wvq started at 2022-11-27 23:39:33 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:34.034: INFO: Container agnhost ready: true, restart count 0 Nov 27 23:40:34.034: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:40:34.034: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:40:34.034: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:39:14 +0000 UTC (0+7 container statuses recorded) Nov 27 23:40:34.034: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:34.034: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:40:34.034: INFO: inline-volume-tester-lnlgr started at 2022-11-27 23:39:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:34.035: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:40:36.085: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:40:36.085: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:40:36.229: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 34164 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:40:36.231: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:40:36.378: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:40:36.526: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:40:36.527: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:40:36.527: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:40:36.527: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:40:36.527: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:40:36.527: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:40:36.527: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:40:36.527: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:40:36.527: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:40:36.527: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:40:36.527: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:40:36.527: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:40:36.527: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:40:36.527: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:36.527: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:36.527: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:40:37.036: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:40:37.036: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:40:37.180: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 39463 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:39:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:40:37.181: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:40:37.329: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:40:37.476: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:40:37.476: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:40:37.476: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:40:37.476: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:40:37.476: INFO: httpd started at 2022-11-27 23:40:13 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:37.476: INFO: Container httpd ready: true, restart count 0 Nov 27 23:40:37.476: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:40:37.476: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:40:37.476: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:40:37.476: INFO: test-rolling-update-with-lb-5f85c5c5f4-rx27j started at 2022-11-27 23:39:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:37.476: INFO: Container agnhost ready: true, restart count 0 Nov 27 23:40:37.476: INFO: ss2-2 started at 2022-11-27 23:40:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:37.476: INFO: Container webserver ready: true, restart count 0 Nov 27 23:40:37.476: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-qszh4 started at 2022-11-27 23:40:21 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:37.476: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:40:37.476: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-bszwm started at 2022-11-27 23:40:25 +0000 UTC (0+1 container statuses recorded) Nov 27 23:40:37.476: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:40:37.983: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:40:37.984: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-6961" for this suite.
Find pod-7a5c1990-6fed-41ae-99a5-85152ba8b5b2 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 27 23:33:49.740: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc004a42f10>: { s: "pod \"pod-148433fe-08a9-4e32-a67a-471f5f20ce44\" is not Running: timed out waiting for the condition", } pod "pod-148433fe-08a9-4e32-a67a-471f5f20ce44" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_07.xml
{"msg":"FAILED External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":-1,"completed":9,"skipped":22,"failed":1,"failures":["External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]} [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:28:47.998: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 27 23:28:48.718: INFO: Creating resource for dynamic PV Nov 27 23:28:48.718: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-2919-e2e-scth4kv �[1mSTEP�[0m: creating a claim Nov 27 23:28:48.863: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 27 23:33:49.740: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc004a42f10>: { s: "pod \"pod-148433fe-08a9-4e32-a67a-471f5f20ce44\" is not Running: timed out waiting for the condition", } pod "pod-148433fe-08a9-4e32-a67a-471f5f20ce44" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000ab1200) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000ab1200) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000ab1200, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 27 23:33:49.740: INFO: Deleting pod "pod-148433fe-08a9-4e32-a67a-471f5f20ce44" in namespace "volume-expand-2919" Nov 27 23:33:49.890: INFO: Wait up to 5m0s for pod "pod-148433fe-08a9-4e32-a67a-471f5f20ce44" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 27 23:33:50.034: INFO: Deleting pod "pod-148433fe-08a9-4e32-a67a-471f5f20ce44" in namespace "volume-expand-2919" �[1mSTEP�[0m: Deleting pvc Nov 27 23:33:50.325: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comznwb7" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-2919". �[1mSTEP�[0m: Found 6 events. Nov 27 23:33:50.766: INFO: At 2022-11-27 23:28:48 +0000 UTC - event for ebs.csi.aws.comznwb7: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:33:50.766: INFO: At 2022-11-27 23:28:49 +0000 UTC - event for ebs.csi.aws.comznwb7: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:33:50.766: INFO: At 2022-11-27 23:28:49 +0000 UTC - event for ebs.csi.aws.comznwb7: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "volume-expand-2919/ebs.csi.aws.comznwb7" Nov 27 23:33:50.766: INFO: At 2022-11-27 23:28:59 +0000 UTC - event for ebs.csi.aws.comznwb7: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-2919-e2e-scth4kv": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:33:50.766: INFO: At 2022-11-27 23:32:06 +0000 UTC - event for ebs.csi.aws.comznwb7: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-2919-e2e-scth4kv": rpc error: code = Aborted desc = Create volume request for pvc-a13365eb-b6c2-461b-be1b-4d56e8df89f1 is already in progress Nov 27 23:33:50.766: INFO: At 2022-11-27 23:33:50 +0000 UTC - event for pod-148433fe-08a9-4e32-a67a-471f5f20ce44: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "pod-148433fe-08a9-4e32-a67a-471f5f20ce44" not found Nov 27 23:33:50.910: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:33:50.910: INFO: Nov 27 23:33:51.057: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:33:51.201: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 27223 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6909":"ip-172-20-33-127.sa-east-1.compute.internal","ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:33:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:33:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6909^e19297f6-6eab-11ed-93e8-a23e43185755],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6909^e19297f6-6eab-11ed-93e8-a23e43185755,DevicePath:,},},Config:nil,},} Nov 27 23:33:51.201: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:33:51.348: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:33:51.497: INFO: service-headless-toggled-jbrsq started at 2022-11-27 23:33:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container service-headless-toggled ready: true, restart count 0 Nov 27 23:33:51.498: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-prn9t started at 2022-11-27 23:33:39 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:51.498: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-wqlvx started at <nil> (0+0 container statuses recorded) Nov 27 23:33:51.498: INFO: service-headless-5lhgs started at 2022-11-27 23:32:59 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container service-headless ready: true, restart count 0 Nov 27 23:33:51.498: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:51.498: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:33:19 +0000 UTC (0+7 container statuses recorded) Nov 27 23:33:51.498: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:51.498: INFO: inline-volume-tester-62hgv started at 2022-11-27 23:33:23 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:33:51.498: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-4rcmw started at 2022-11-27 23:33:37 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:51.498: INFO: ss2-0 started at 2022-11-27 23:33:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container webserver ready: false, restart count 0 Nov 27 23:33:51.498: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:51.498: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:51.498: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:51.498: INFO: pod-bf5ef77b-880f-48b2-9036-fc97bf9ab016 started at 2022-11-27 23:33:46 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:51.498: INFO: Container write-pod ready: true, restart count 0 Nov 27 23:33:52.040: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:33:52.040: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:52.185: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 27222 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:30:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:33:52.185: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:52.338: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:52.490: INFO: test-pod started at 2022-11-27 23:33:44 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:52.490: INFO: Container busybox-1 ready: true, restart count 0 Nov 27 23:33:52.490: INFO: Container busybox-2 ready: true, restart count 0 Nov 27 23:33:52.490: INFO: Container busybox-3 ready: true, restart count 0 Nov 27 23:33:52.490: INFO: hostexec-ip-172-20-34-252.sa-east-1.compute.internal-tnlxp started at 2022-11-27 23:33:45 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:52.490: INFO: service-headless-toggled-hp7x9 started at 2022-11-27 23:33:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container service-headless-toggled ready: true, restart count 0 Nov 27 23:33:52.490: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:52.490: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:52.490: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:33:52.490: INFO: service-headless-4cghp started at 2022-11-27 23:32:59 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container service-headless ready: true, restart count 0 Nov 27 23:33:52.490: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container coredns ready: true, restart count 0 Nov 27 23:33:52.490: INFO: boom-server started at 2022-11-27 23:32:34 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container boom-server ready: true, restart count 0 Nov 27 23:33:52.490: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:52.490: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:52.490: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:52.490: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:52.490: INFO: test-rs-xfh5z started at 2022-11-27 23:33:50 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container httpd ready: true, restart count 0 Nov 27 23:33:52.490: INFO: ss2-2 started at 2022-11-27 23:33:52 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:52.490: INFO: Container webserver ready: false, restart count 0 Nov 27 23:33:53.023: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:53.023: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:53.167: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 27739 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5756":"ip-172-20-50-151.sa-east-1.compute.internal","csi-mock-csi-mock-volumes-8556":"csi-mock-csi-mock-volumes-8556","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:28:17 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:18 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:18 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:18 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:18 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-5756^d6ab5857-6eab-11ed-ace9-9299817b0c2d kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-5756^d6ab5857-6eab-11ed-ace9-9299817b0c2d,DevicePath:,},},Config:nil,},} Nov 27 23:33:53.168: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:53.322: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:53.477: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:53.477: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:53.477: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:53.477: INFO: Container coredns ready: true, restart count 0 Nov 27 23:33:53.477: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:31:28 +0000 UTC (0+7 container statuses recorded) Nov 27 23:33:53.477: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:53.477: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:53.477: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:53.477: INFO: csi-mockplugin-0 started at 2022-11-27 23:33:42 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:53.477: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:33:53.477: INFO: Container mock ready: true, restart count 0 Nov 27 23:33:53.477: INFO: pvc-volume-tester-8tz9w started at 2022-11-27 23:33:48 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:53.477: INFO: Container volume-tester ready: true, restart count 0 Nov 27 23:33:53.478: INFO: inline-volume-tester-xbmcg started at 2022-11-27 23:33:11 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:53.478: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:33:54.014: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:54.014: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:54.159: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 25553 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:33:54.159: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:54.307: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:54.461: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:54.461: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:54.461: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:33:54.461: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:33:54.461: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:33:54.461: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:33:54.461: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:54.461: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:33:54.461: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:54.461: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:33:54.461: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:33:54.461: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:33:54.461: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:33:54.461: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:33:54.461: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:54.461: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:33:54.993: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:54.993: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:55.138: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 26694 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:30:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:33:55.138: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:55.286: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:55.434: INFO: pod-configmaps-41b9ee93-1531-4ece-b759-afec8078dd01 started at <nil> (0+0 container statuses recorded) Nov 27 23:33:55.434: INFO: test-rs-kwjt8 started at 2022-11-27 23:33:52 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:55.434: INFO: Container httpd ready: false, restart count 0 Nov 27 23:33:55.434: INFO: test-rs-76gj7 started at <nil> (0+0 container statuses recorded) Nov 27 23:33:55.434: INFO: test-host-network-pod started at 2022-11-27 23:33:49 +0000 UTC (0+2 container statuses recorded) Nov 27 23:33:55.434: INFO: Container busybox-1 ready: true, restart count 0 Nov 27 23:33:55.434: INFO: Container busybox-2 ready: true, restart count 0 Nov 27 23:33:55.434: INFO: startup-script started at 2022-11-27 23:33:11 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:55.434: INFO: Container startup-script ready: true, restart count 0 Nov 27 23:33:55.434: INFO: service-headless-toggled-5cxnn started at 2022-11-27 23:33:12 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:55.434: INFO: Container service-headless-toggled ready: true, restart count 0 Nov 27 23:33:55.434: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:55.434: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:55.434: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:55.434: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:55.434: INFO: service-headless-pt5zc started at 2022-11-27 23:32:59 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:55.434: INFO: Container service-headless ready: true, restart count 0 Nov 27 23:33:55.434: INFO: test-cleanup-controller-8hz5j started at <nil> (0+0 container statuses recorded) Nov 27 23:33:55.434: INFO: ss2-1 started at 2022-11-27 23:33:29 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:55.434: INFO: Container webserver ready: true, restart count 0 Nov 27 23:33:55.434: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:55.434: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:55.434: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:56.040: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:56.040: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-2919" for this suite.
Find pod-148433fe-08a9-4e32-a67a-471f5f20ce44 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:28:43.838: Unexpected error: <*errors.errorString | 0xc0042a0b10>: { s: "pod \"pod-63d5dd78-d342-42da-ba6d-0d3fcd347d3d\" is not Running: timed out waiting for the condition", } pod "pod-63d5dd78-d342-42da-ba6d-0d3fcd347d3d" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_17.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:23:42.113: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:23:42.828: INFO: Creating resource for dynamic PV Nov 27 23:23:42.828: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-9628-e2e-scm8ds5 �[1mSTEP�[0m: creating a claim Nov 27 23:23:42.973: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-9628 with fsgroup 1000 Nov 27 23:28:43.838: FAIL: Unexpected error: <*errors.errorString | 0xc0042a0b10>: { s: "pod \"pod-63d5dd78-d342-42da-ba6d-0d3fcd347d3d\" is not Running: timed out waiting for the condition", } pod "pod-63d5dd78-d342-42da-ba6d-0d3fcd347d3d" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc00409fa20, 0xc0039f70d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00037b380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00037b380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00037b380, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 27 23:28:43.981: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comlkgp8" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-9628". �[1mSTEP�[0m: Found 5 events. Nov 27 23:28:44.411: INFO: At 2022-11-27 23:23:43 +0000 UTC - event for ebs.csi.aws.comlkgp8: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:28:44.411: INFO: At 2022-11-27 23:23:43 +0000 UTC - event for ebs.csi.aws.comlkgp8: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:28:44.411: INFO: At 2022-11-27 23:23:43 +0000 UTC - event for ebs.csi.aws.comlkgp8: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-9628/ebs.csi.aws.comlkgp8" Nov 27 23:28:44.411: INFO: At 2022-11-27 23:23:53 +0000 UTC - event for ebs.csi.aws.comlkgp8: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-9628-e2e-scm8ds5": rpc error: code = Internal desc = Could not create volume "pvc-55d0602e-9fe2-4d54-850d-89f103f3a409": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:28:44.411: INFO: At 2022-11-27 23:24:04 +0000 UTC - event for ebs.csi.aws.comlkgp8: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-9628-e2e-scm8ds5": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:28:44.554: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:28:44.554: INFO: pod-63d5dd78-d342-42da-ba6d-0d3fcd347d3d Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:28:44 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-9628/ebs.csi.aws.comlkgp8"}] Nov 27 23:28:44.554: INFO: Nov 27 23:28:44.698: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:44.851: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 17010 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:27:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:44.852: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:44.997: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:45.151: INFO: pfpod started at 2022-11-27 23:28:42 +0000 UTC (0+2 container statuses recorded) Nov 27 23:28:45.151: INFO: Container portforwardtester ready: true, restart count 0 Nov 27 23:28:45.151: INFO: Container readiness ready: false, restart count 0 Nov 27 23:28:45.151: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:45.151: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:45.151: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:45.151: INFO: adopt-release-p7wf7 started at 2022-11-27 23:28:22 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.151: INFO: Container c ready: true, restart count 0 Nov 27 23:28:45.151: INFO: test-container-pod started at 2022-11-27 23:28:39 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.151: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:45.151: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-wnj2b started at 2022-11-27 23:28:43 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.151: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:28:45.151: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:45.151: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:45.151: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:45.151: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:45.151: INFO: netserver-0 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.151: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:45.151: INFO: test-rolling-update-controller-mz72q started at 2022-11-27 23:28:40 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.151: INFO: Container httpd ready: true, restart count 0 Nov 27 23:28:45.651: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:45.651: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:45.794: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 17005 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:24:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:45.794: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:45.939: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:46.086: INFO: pfpod started at 2022-11-27 23:28:40 +0000 UTC (0+2 container statuses recorded) Nov 27 23:28:46.086: INFO: Container portforwardtester ready: true, restart count 0 Nov 27 23:28:46.086: INFO: Container readiness ready: false, restart count 0 Nov 27 23:28:46.086: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:46.086: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:46.086: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:28:46.086: INFO: test-rolling-update-deployment-54955647cf-9qhqh started at 2022-11-27 23:28:44 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container agnhost ready: false, restart count 0 Nov 27 23:28:46.086: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container coredns ready: true, restart count 0 Nov 27 23:28:46.086: INFO: netserver-1 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:46.086: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:46.086: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:46.086: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:46.086: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:46.086: INFO: hostpath-symlink-prep-provisioning-5865 started at 2022-11-27 23:28:45 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container init-volume-provisioning-5865 ready: false, restart count 0 Nov 27 23:28:46.086: INFO: busybox-privileged-false-e5632c40-ea1d-4dcf-8c43-963b9f0a6942 started at 2022-11-27 23:28:39 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container busybox-privileged-false-e5632c40-ea1d-4dcf-8c43-963b9f0a6942 ready: false, restart count 0 Nov 27 23:28:46.086: INFO: adopt-release-k4hvs started at 2022-11-27 23:28:15 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.086: INFO: Container c ready: true, restart count 0 Nov 27 23:28:46.578: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:46.578: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:46.722: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 17932 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8085":"ip-172-20-50-151.sa-east-1.compute.internal","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:28:17 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2b5f598c-6eab-11ed-8b02-be523c12f2cd kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8085^2b5f598c-6eab-11ed-8b02-be523c12f2cd,DevicePath:,},},Config:nil,},} Nov 27 23:28:46.723: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:46.869: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:47.018: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:28:14 +0000 UTC (0+7 container statuses recorded) Nov 27 23:28:47.018: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:47.018: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:47.018: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:47.018: INFO: netserver-2 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.018: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:47.018: INFO: inline-volume-tester2-69klq started at 2022-11-27 23:28:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.018: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:28:47.018: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.018: INFO: Container coredns ready: true, restart count 0 Nov 27 23:28:47.018: INFO: inline-volume-tester-sfkx4 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.018: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:28:47.018: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:47.018: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:47.018: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:47.542: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:47.542: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:47.686: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 15987 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:47.686: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:47.832: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:47.980: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:28:47.980: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:28:47.980: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:28:47.980: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:28:47.980: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:47.980: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:47.980: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:47.980: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:28:47.980: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:28:47.980: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:28:47.980: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:28:47.980: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.980: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:28:47.980: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:28:47.980: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:47.980: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:48.480: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:48.480: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:48.624: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 18398 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8564":"csi-mock-csi-mock-volumes-8564","ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:28:47 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:48.625: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:48.771: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:48.918: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:48.918: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:48.918: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:48.918: INFO: busybox-8c290bb6-883a-4503-b7b8-48cf81b29b5a started at 2022-11-27 23:26:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:48.918: INFO: Container busybox ready: true, restart count 0 Nov 27 23:28:48.918: INFO: adopt-release-trt5r started at 2022-11-27 23:28:15 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:48.918: INFO: Container c ready: true, restart count 0 Nov 27 23:28:48.918: INFO: csi-mockplugin-attacher-0 started at 2022-11-27 23:28:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:48.919: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:28:48.919: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:48.919: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:48.919: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:48.919: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:48.919: INFO: test-webserver-48243c59-6a81-4fab-8553-ae5f710c7444 started at 2022-11-27 23:25:01 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:48.919: INFO: Container test-webserver ready: true, restart count 0 Nov 27 23:28:48.919: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-2gspb started at 2022-11-27 23:28:45 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:48.919: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:28:48.919: INFO: liveness-33e27314-ba60-45c8-9252-4faa7e734eaf started at 2022-11-27 23:25:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:48.919: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:28:48.919: INFO: csi-mockplugin-0 started at 2022-11-27 23:28:24 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:48.919: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:28:48.919: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:28:48.919: INFO: Container mock ready: true, restart count 0 Nov 27 23:28:49.417: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:49.417: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-9628" for this suite.
Find pod-63d5dd78-d342-42da-ba6d-0d3fcd347d3d mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:41:24.607: Unexpected error: <*errors.errorString | 0xc002c7a600>: { s: "pod \"pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4\" is not Running: timed out waiting for the condition", } pod "pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:36:22.865: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:36:23.590: INFO: Creating resource for dynamic PV Nov 27 23:36:23.590: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-7293-e2e-scmpp9b �[1mSTEP�[0m: creating a claim Nov 27 23:36:23.736: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-7293 with fsgroup 1000 Nov 27 23:41:24.607: FAIL: Unexpected error: <*errors.errorString | 0xc002c7a600>: { s: "pod \"pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4\" is not Running: timed out waiting for the condition", } pod "pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0043cdb80, 0xc0021a50d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000be4900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000be4900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000be4900, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 27 23:41:24.758: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comjtgtj" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-7293". �[1mSTEP�[0m: Found 5 events. Nov 27 23:41:25.207: INFO: At 2022-11-27 23:36:23 +0000 UTC - event for ebs.csi.aws.comjtgtj: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:41:25.207: INFO: At 2022-11-27 23:36:24 +0000 UTC - event for ebs.csi.aws.comjtgtj: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-7293/ebs.csi.aws.comjtgtj" Nov 27 23:41:25.207: INFO: At 2022-11-27 23:36:24 +0000 UTC - event for ebs.csi.aws.comjtgtj: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:41:25.207: INFO: At 2022-11-27 23:36:34 +0000 UTC - event for ebs.csi.aws.comjtgtj: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-7293-e2e-scmpp9b": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:41:25.207: INFO: At 2022-11-27 23:41:25 +0000 UTC - event for pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-7293/ebs.csi.aws.comjtgtj" Nov 27 23:41:25.351: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:41:25.351: INFO: pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4 Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:41:25 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-7293/ebs.csi.aws.comjtgtj"}] Nov 27 23:41:25.351: INFO: Nov 27 23:41:25.497: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:41:25.642: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 38644 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:37:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:41:25.643: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:41:25.790: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:41:25.938: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:41:25.938: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:41:25.938: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:41:25.938: INFO: deployment-585449566-p56rv started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:25.938: INFO: Container nginx ready: false, restart count 0 Nov 27 23:41:25.938: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-mv88s started at 2022-11-27 23:41:20 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:25.938: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:41:25.938: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:41:25.938: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:41:25.938: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:41:25.938: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:41:25.938: INFO: my-hostname-basic-931d8c9e-b112-4332-bb4d-705ca3c02fc8-nzhdt started at 2022-11-27 23:41:09 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:25.938: INFO: Container my-hostname-basic-931d8c9e-b112-4332-bb4d-705ca3c02fc8 ready: true, restart count 0 Nov 27 23:41:25.938: INFO: probe-test-9884113b-b5fe-4b0d-9a17-cca7b3e1c478 started at 2022-11-27 23:41:15 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:25.938: INFO: Container probe-test-9884113b-b5fe-4b0d-9a17-cca7b3e1c478 ready: true, restart count 0 Nov 27 23:41:26.441: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:41:26.441: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:41:26.585: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 40521 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6525":"csi-mock-csi-mock-volumes-6525","ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:37:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:41:26.585: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:41:26.733: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:41:26.882: INFO: pod-configmaps-cae28f30-935e-4fce-bfda-cf43486f39fa started at 2022-11-27 23:41:25 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Container agnhost-container ready: false, restart count 0 Nov 27 23:41:26.882: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:41:26.882: INFO: all-succeed-l8pjc started at 2022-11-27 23:41:23 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Container c ready: false, restart count 0 Nov 27 23:41:26.882: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:41:26.882: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:41:26.882: INFO: deployment-55649fd747-jsxmh started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Container nginx ready: false, restart count 0 Nov 27 23:41:26.882: INFO: pod-terminate-status-2-5 started at 2022-11-27 23:41:25 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Container fail ready: false, restart count 0 Nov 27 23:41:26.882: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:26.882: INFO: Container coredns ready: true, restart count 0 Nov 27 23:41:26.882: INFO: csi-mockplugin-0 started at 2022-11-27 23:41:05 +0000 UTC (0+4 container statuses recorded) Nov 27 23:41:26.882: INFO: Container busybox ready: true, restart count 0 Nov 27 23:41:26.882: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:41:26.882: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:41:26.882: INFO: Container mock ready: true, restart count 0 Nov 27 23:41:26.882: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:41:26.882: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:41:26.882: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:41:26.882: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:41:27.762: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:41:27.762: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:41:27.914: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 39938 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:40:39 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:41:27.914: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:41:28.061: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:41:28.212: INFO: hostexec-ip-172-20-50-151.sa-east-1.compute.internal-rzqv5 started at 2022-11-27 23:41:21 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:41:28.212: INFO: concurrent-27826541-ntmvw started at 2022-11-27 23:41:00 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container c ready: true, restart count 0 Nov 27 23:41:28.212: INFO: proxy-service-j9fbb-2m4c2 started at 2022-11-27 23:41:21 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container proxy-service-j9fbb ready: false, restart count 0 Nov 27 23:41:28.212: INFO: deployment-585449566-42q9m started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container nginx ready: false, restart count 0 Nov 27 23:41:28.212: INFO: all-succeed-74x2c started at <nil> (0+0 container statuses recorded) Nov 27 23:41:28.212: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:41:28.212: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:41:28.212: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container coredns ready: true, restart count 0 Nov 27 23:41:28.212: INFO: all-succeed-m8nv4 started at 2022-11-27 23:41:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container c ready: false, restart count 0 Nov 27 23:41:28.212: INFO: all-succeed-tjrfg started at 2022-11-27 23:41:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container c ready: false, restart count 0 Nov 27 23:41:28.212: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:41:28.212: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:41:28.212: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:41:28.212: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:41:28.212: INFO: run-test-3 started at 2022-11-27 23:40:58 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:28.212: INFO: Container run-test-3 ready: true, restart count 0 Nov 27 23:41:30.152: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:41:30.152: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:41:30.299: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 34164 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:37:22 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:41:30.299: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:41:30.450: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:41:30.604: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:41:30.605: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:41:30.605: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:41:30.605: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:41:30.605: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:41:30.605: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:41:30.605: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:41:30.605: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:41:30.605: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:41:30.605: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:41:30.605: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:41:30.605: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:41:30.605: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:41:30.605: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:41:30.605: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:30.605: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:41:31.160: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:41:31.160: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:41:31.305: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 39463 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:39:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:39:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:41:31.306: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:41:31.454: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:41:31.601: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-5zlzs started at 2022-11-27 23:41:00 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:41:31.601: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:41:31.601: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:41:31.601: INFO: pod-subpath-test-preprovisionedpv-xlsh started at 2022-11-27 23:41:11 +0000 UTC (1+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Init container init-volume-preprovisionedpv-xlsh ready: true, restart count 0 Nov 27 23:41:31.601: INFO: Container test-container-subpath-preprovisionedpv-xlsh ready: true, restart count 0 Nov 27 23:41:31.601: INFO: deployment-585449566-kkvn4 started at 2022-11-27 23:41:18 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Container nginx ready: false, restart count 0 Nov 27 23:41:31.601: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:41:31.601: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:41:31.601: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:41:31.601: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:41:31.601: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-wbxsb started at 2022-11-27 23:40:50 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:41:31.601: INFO: pod-terminate-status-1-5 started at <nil> (0+0 container statuses recorded) Nov 27 23:41:31.601: INFO: local-client started at 2022-11-27 23:41:26 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Container local-client ready: true, restart count 0 Nov 27 23:41:31.601: INFO: pod-terminate-status-0-5 started at 2022-11-27 23:41:27 +0000 UTC (0+1 container statuses recorded) Nov 27 23:41:31.601: INFO: Container fail ready: false, restart count 0 Nov 27 23:41:33.635: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:41:33.636: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-7293" for this suite.
Find pod-2965c5aa-ed23-4c22-a0eb-e062979de9b4 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:26:49.117: Unexpected error: <*errors.errorString | 0xc00335ece0>: { s: "pod \"pod-6b65ecbb-07b3-4258-8834-87de3dc53c77\" is not Running: timed out waiting for the condition", } pod "pod-6b65ecbb-07b3-4258-8834-87de3dc53c77" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_24.xml
{"msg":"PASSED [sig-network] Ingress API should support creating Ingress API operations [Conformance]","total":-1,"completed":7,"skipped":58,"failed":0} [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:21:47.317: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:21:48.041: INFO: Creating resource for dynamic PV Nov 27 23:21:48.041: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-5002-e2e-scm2ld7 �[1mSTEP�[0m: creating a claim Nov 27 23:21:48.250: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-5002 with fsgroup 1000 Nov 27 23:26:49.117: FAIL: Unexpected error: <*errors.errorString | 0xc00335ece0>: { s: "pod \"pod-6b65ecbb-07b3-4258-8834-87de3dc53c77\" is not Running: timed out waiting for the condition", } pod "pod-6b65ecbb-07b3-4258-8834-87de3dc53c77" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0021ff340, 0xc00420d0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001cd680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0001cd680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0001cd680, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 27 23:26:49.261: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com67fz4" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-5002". �[1mSTEP�[0m: Found 5 events. Nov 27 23:26:49.695: INFO: At 2022-11-27 23:21:48 +0000 UTC - event for ebs.csi.aws.com67fz4: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:26:49.695: INFO: At 2022-11-27 23:21:48 +0000 UTC - event for ebs.csi.aws.com67fz4: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:26:49.695: INFO: At 2022-11-27 23:21:48 +0000 UTC - event for ebs.csi.aws.com67fz4: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-5002/ebs.csi.aws.com67fz4" Nov 27 23:26:49.695: INFO: At 2022-11-27 23:21:58 +0000 UTC - event for ebs.csi.aws.com67fz4: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5002-e2e-scm2ld7": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:26:49.695: INFO: At 2022-11-27 23:22:09 +0000 UTC - event for ebs.csi.aws.com67fz4: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5002-e2e-scm2ld7": rpc error: code = Internal desc = Could not create volume "pvc-a712f2ae-040c-4ada-8f4b-16ae1d6083fc": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:26:49.839: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:26:49.839: INFO: pod-6b65ecbb-07b3-4258-8834-87de3dc53c77 Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:26:49 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-5002/ebs.csi.aws.com67fz4"}] Nov 27 23:26:49.839: INFO: Nov 27 23:26:49.984: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:26:50.127: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 14872 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-2227":"ip-172-20-33-127.sa-east-1.compute.internal","csi-hostpath-volume-expand-2971":"ip-172-20-33-127.sa-east-1.compute.internal","ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:26:29 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:26:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:26:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-2227^eb2d068b-6eaa-11ed-a7a0-7670240bd8e5],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-2227^eb2d068b-6eaa-11ed-a7a0-7670240bd8e5,DevicePath:,},},Config:nil,},} Nov 27 23:26:50.128: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:26:50.274: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:26:50.433: INFO: busybox-d4f22875-c0a9-4ee0-998a-58f7e95305c6 started at 2022-11-27 23:26:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Container busybox ready: true, restart count 0 Nov 27 23:26:50.433: INFO: hostexec-ip-172-20-33-127.sa-east-1.compute.internal-clqp7 started at 2022-11-27 23:26:37 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:26:50.433: INFO: inline-volume-tester-hpv8b started at 2022-11-27 23:26:29 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:26:50.433: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:50.433: INFO: pod-0a6830b0-e973-47f0-bb1d-bbec9995cf68 started at 2022-11-27 23:26:46 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Container write-pod ready: true, restart count 0 Nov 27 23:26:50.433: INFO: netserver-0 started at 2022-11-27 23:26:41 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Container webserver ready: false, restart count 0 Nov 27 23:26:50.433: INFO: concurrent-27826526-8xh89 started at 2022-11-27 23:26:00 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:50.433: INFO: Container c ready: false, restart count 0 Nov 27 23:26:50.433: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:26:25 +0000 UTC (0+7 container statuses recorded) Nov 27 23:26:50.433: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:50.433: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:50.433: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:50.433: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:26:02 +0000 UTC (0+7 container statuses recorded) Nov 27 23:26:50.433: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:50.433: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:51.007: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:26:51.007: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:26:51.151: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 14286 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:24:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:24:47 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:26:51.151: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:26:51.297: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:26:51.446: INFO: netserver-1 started at 2022-11-27 23:26:41 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:51.446: INFO: Container webserver ready: false, restart count 0 Nov 27 23:26:51.446: INFO: pod-1 started at 2022-11-27 23:26:33 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:51.446: INFO: Container donothing ready: true, restart count 0 Nov 27 23:26:51.446: INFO: pod-logs-websocket-83b2e194-94da-4cd0-ac75-98bd6d4d7355 started at 2022-11-27 23:26:50 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:51.446: INFO: Container main ready: false, restart count 0 Nov 27 23:26:51.446: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:51.446: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:51.446: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:51.446: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:51.446: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:51.446: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:26:51.446: INFO: dns-test-96a9d51d-7615-4c1e-9782-75bead794047 started at 2022-11-27 23:26:46 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:51.446: INFO: Container jessie-querier ready: false, restart count 0 Nov 27 23:26:51.446: INFO: Container querier ready: false, restart count 0 Nov 27 23:26:51.446: INFO: Container webserver ready: false, restart count 0 Nov 27 23:26:51.446: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:51.446: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:51.446: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:51.446: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:51.446: INFO: Container coredns ready: true, restart count 0 Nov 27 23:26:51.963: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:26:51.963: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:52.107: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 15177 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4834":"ip-172-20-50-151.sa-east-1.compute.internal","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:26:42 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:26:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:48 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:48 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:48 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:26:48 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-4834^f2ea5926-6eaa-11ed-ab44-7a5c98a2f0a7 kubernetes.io/csi/csi-hostpath-ephemeral-4834^f2ea7f29-6eaa-11ed-ab44-7a5c98a2f0a7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-4834^f2ea5926-6eaa-11ed-ab44-7a5c98a2f0a7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-4834^f2ea7f29-6eaa-11ed-ab44-7a5c98a2f0a7,DevicePath:,},},Config:nil,},} Nov 27 23:26:52.108: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:52.254: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:52.407: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:52.407: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:52.407: INFO: netserver-2 started at 2022-11-27 23:26:41 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:52.407: INFO: Container webserver ready: false, restart count 0 Nov 27 23:26:52.407: INFO: inline-volume-tester-jh9bj started at 2022-11-27 23:26:42 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:52.407: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:26:52.407: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:52.407: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:52.407: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:52.407: INFO: Container coredns ready: true, restart count 0 Nov 27 23:26:52.407: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:26:39 +0000 UTC (0+7 container statuses recorded) Nov 27 23:26:52.407: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:52.407: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:52.922: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:26:52.923: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:53.067: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 6150 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:22:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:26:53.067: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:53.218: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:53.508: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:26:53.508: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:26:53.508: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:53.508: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:26:53.508: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:26:53.508: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:26:53.508: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:26:53.508: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:53.508: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:53.508: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:53.508: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:53.508: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:26:53.508: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:26:53.508: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:26:53.508: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:53.508: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:26:54.004: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:26:54.004: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:54.148: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 14587 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:16 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:16 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:26:16 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:26:16 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:26:54.148: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:54.294: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:54.442: INFO: netserver-3 started at 2022-11-27 23:26:41 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:54.442: INFO: Container webserver ready: false, restart count 0 Nov 27 23:26:54.442: INFO: liveness-33e27314-ba60-45c8-9252-4faa7e734eaf started at 2022-11-27 23:25:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:54.442: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:26:54.442: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:26:54.443: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:26:54.443: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:26:54.444: INFO: busybox-8c290bb6-883a-4503-b7b8-48cf81b29b5a started at 2022-11-27 23:26:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:54.444: INFO: Container busybox ready: true, restart count 0 Nov 27 23:26:54.444: INFO: pod-0 started at 2022-11-27 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:54.444: INFO: Container donothing ready: true, restart count 0 Nov 27 23:26:54.444: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:26:54.444: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:26:54.444: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:26:54.444: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:26:54.444: INFO: test-webserver-48243c59-6a81-4fab-8553-ae5f710c7444 started at 2022-11-27 23:25:01 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:54.444: INFO: Container test-webserver ready: true, restart count 0 Nov 27 23:26:54.444: INFO: pod-2 started at 2022-11-27 23:26:33 +0000 UTC (0+1 container statuses recorded) Nov 27 23:26:54.444: INFO: Container donothing ready: true, restart count 0 Nov 27 23:26:54.938: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:26:54.938: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-5002" for this suite.
Find pod-6b65ecbb-07b3-4258-8834-87de3dc53c77 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:46:21.575: Unexpected error: <*errors.errorString | 0xc003ecc870>: { s: "pod \"pod-8998deba-7bde-44ed-a21d-e1115426c3dc\" is not Running: timed out waiting for the condition", } pod "pod-8998deba-7bde-44ed-a21d-e1115426c3dc" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_09.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:41:19.833: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:41:20.559: INFO: Creating resource for dynamic PV Nov 27 23:41:20.559: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-3451-e2e-scmsm2w �[1mSTEP�[0m: creating a claim Nov 27 23:41:20.704: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-3451 with fsgroup 1000 Nov 27 23:46:21.574: FAIL: Unexpected error: <*errors.errorString | 0xc003ecc870>: { s: "pod \"pod-8998deba-7bde-44ed-a21d-e1115426c3dc\" is not Running: timed out waiting for the condition", } pod "pod-8998deba-7bde-44ed-a21d-e1115426c3dc" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001f974a0, 0xc001c010d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00020b680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00020b680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00020b680, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 27 23:46:21.719: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comnr2mt" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-3451". �[1mSTEP�[0m: Found 5 events. Nov 27 23:46:22.154: INFO: At 2022-11-27 23:41:20 +0000 UTC - event for ebs.csi.aws.comnr2mt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:46:22.154: INFO: At 2022-11-27 23:41:21 +0000 UTC - event for ebs.csi.aws.comnr2mt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:46:22.154: INFO: At 2022-11-27 23:41:21 +0000 UTC - event for ebs.csi.aws.comnr2mt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-3451/ebs.csi.aws.comnr2mt" Nov 27 23:46:22.154: INFO: At 2022-11-27 23:41:31 +0000 UTC - event for ebs.csi.aws.comnr2mt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-3451-e2e-scmsm2w": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:46:22.154: INFO: At 2022-11-27 23:46:22 +0000 UTC - event for pod-8998deba-7bde-44ed-a21d-e1115426c3dc: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-3451/ebs.csi.aws.comnr2mt" Nov 27 23:46:22.298: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:46:22.298: INFO: pod-8998deba-7bde-44ed-a21d-e1115426c3dc Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:46:22 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-3451/ebs.csi.aws.comnr2mt"}] Nov 27 23:46:22.298: INFO: Nov 27 23:46:22.444: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:46:22.588: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 48150 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:45:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:45:48 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:45:48 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:45:48 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:45:48 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:46:22.589: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:46:22.735: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:46:22.881: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:46:22.881: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:46:22.881: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:46:22.881: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:46:22.881: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:46:22.881: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:46:22.881: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:46:23.393: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:46:23.393: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:46:23.537: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 46927 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:43:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:43:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:43:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:43:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:43:58 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:46:23.537: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:46:23.685: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:46:23.832: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:23.832: INFO: Container coredns ready: true, restart count 0 Nov 27 23:46:23.832: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:46:23.832: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:46:23.832: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:46:23.832: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:46:23.832: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:46:23.832: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:46:23.832: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:46:23.832: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:23.832: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:46:24.331: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:46:24.331: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:46:24.476: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 47310 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-5702 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {e2e.test Update v1 2022-11-27 23:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:44:09 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:44:09 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:44:09 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:44:09 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:46:24.477: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:46:24.624: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:46:24.771: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:24.771: INFO: Container coredns ready: true, restart count 0 Nov 27 23:46:24.771: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:46:24.771: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:46:24.771: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:46:24.771: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:46:24.771: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:46:24.771: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:46:24.771: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:46:25.286: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:46:25.286: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:46:25.431: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 43117 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:42:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:42:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:42:22 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:42:22 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:46:25.432: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:46:25.581: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:46:25.730: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:46:25.730: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:46:25.730: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:46:25.730: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:46:25.730: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:46:25.730: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:46:25.730: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:46:25.730: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:46:25.730: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:46:25.730: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:46:25.730: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:46:25.730: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:46:25.730: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:46:25.730: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:46:25.730: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:46:25.730: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:46:26.225: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:46:26.225: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:46:26.370: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 47846 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:39:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:44:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:44:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:44:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:44:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:46:26.370: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:46:26.517: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:46:26.664: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:46:26.664: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:46:26.664: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:46:26.664: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:46:26.664: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:46:26.664: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:46:26.664: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:46:27.154: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:46:27.155: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-3451" for this suite.
Find pod-8998deba-7bde-44ed-a21d-e1115426c3dc mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:28:41.730: Unexpected error: <*errors.errorString | 0xc00297a570>: { s: "pod \"pod-30713434-a567-4be1-865e-086c7884b449\" is not Running: timed out waiting for the condition", } pod "pod-30713434-a567-4be1-865e-086c7884b449" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_07.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:23:39.947: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:23:40.712: INFO: Creating resource for dynamic PV Nov 27 23:23:40.712: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-8649-e2e-scjxx4x �[1mSTEP�[0m: creating a claim Nov 27 23:23:40.859: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-8649 with fsgroup 1000 Nov 27 23:28:41.730: FAIL: Unexpected error: <*errors.errorString | 0xc00297a570>: { s: "pod \"pod-30713434-a567-4be1-865e-086c7884b449\" is not Running: timed out waiting for the condition", } pod "pod-30713434-a567-4be1-865e-086c7884b449" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0043ad760, 0xc0020d70d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000ab1200) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000ab1200) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000ab1200, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 27 23:28:41.875: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comts8dq" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-8649". �[1mSTEP�[0m: Found 6 events. Nov 27 23:28:42.330: INFO: At 2022-11-27 23:23:40 +0000 UTC - event for ebs.csi.aws.comts8dq: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:28:42.330: INFO: At 2022-11-27 23:23:41 +0000 UTC - event for ebs.csi.aws.comts8dq: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:28:42.330: INFO: At 2022-11-27 23:23:41 +0000 UTC - event for ebs.csi.aws.comts8dq: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-8649/ebs.csi.aws.comts8dq" Nov 27 23:28:42.330: INFO: At 2022-11-27 23:23:51 +0000 UTC - event for ebs.csi.aws.comts8dq: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8649-e2e-scjxx4x": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:28:42.330: INFO: At 2022-11-27 23:24:46 +0000 UTC - event for ebs.csi.aws.comts8dq: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8649-e2e-scjxx4x": rpc error: code = Internal desc = Could not create volume "pvc-432c172f-fdee-439e-9507-b5eda7a47ee0": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:28:42.330: INFO: At 2022-11-27 23:28:42 +0000 UTC - event for pod-30713434-a567-4be1-865e-086c7884b449: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-8649/ebs.csi.aws.comts8dq" Nov 27 23:28:42.474: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:28:42.474: INFO: pod-30713434-a567-4be1-865e-086c7884b449 Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:28:42 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-8649/ebs.csi.aws.comts8dq"}] Nov 27 23:28:42.474: INFO: Nov 27 23:28:42.620: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:42.768: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 17010 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:27:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:42.768: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:42.915: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:43.070: INFO: pfpod started at 2022-11-27 23:28:42 +0000 UTC (0+2 container statuses recorded) Nov 27 23:28:43.070: INFO: Container portforwardtester ready: false, restart count 0 Nov 27 23:28:43.070: INFO: Container readiness ready: false, restart count 0 Nov 27 23:28:43.070: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:43.070: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:43.070: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:43.070: INFO: adopt-release-p7wf7 started at 2022-11-27 23:28:22 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:43.070: INFO: Container c ready: true, restart count 0 Nov 27 23:28:43.070: INFO: test-container-pod started at 2022-11-27 23:28:39 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:43.070: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:43.070: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:43.070: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:43.070: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:43.070: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:43.070: INFO: netserver-0 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:43.070: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:43.070: INFO: alpine-nnp-true-328a3ef6-7003-4563-b3a0-35c0098ff6ae started at 2022-11-27 23:28:30 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:43.070: INFO: Container alpine-nnp-true-328a3ef6-7003-4563-b3a0-35c0098ff6ae ready: false, restart count 0 Nov 27 23:28:43.070: INFO: test-rolling-update-controller-mz72q started at 2022-11-27 23:28:40 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:43.070: INFO: Container httpd ready: true, restart count 0 Nov 27 23:28:43.592: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:28:43.592: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:43.737: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 17005 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:24:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:27:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:43.738: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:43.885: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:44.033: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:44.033: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:44.033: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:44.033: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:44.033: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:28:44.033: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:44.033: INFO: Container coredns ready: true, restart count 0 Nov 27 23:28:44.033: INFO: adopt-release-k4hvs started at 2022-11-27 23:28:15 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:44.033: INFO: Container c ready: true, restart count 0 Nov 27 23:28:44.033: INFO: netserver-1 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:44.033: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:44.033: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:44.033: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:44.033: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:44.033: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:44.033: INFO: busybox-privileged-false-e5632c40-ea1d-4dcf-8c43-963b9f0a6942 started at 2022-11-27 23:28:39 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:44.033: INFO: Container busybox-privileged-false-e5632c40-ea1d-4dcf-8c43-963b9f0a6942 ready: false, restart count 0 Nov 27 23:28:44.033: INFO: pfpod started at 2022-11-27 23:28:40 +0000 UTC (0+2 container statuses recorded) Nov 27 23:28:44.033: INFO: Container portforwardtester ready: true, restart count 0 Nov 27 23:28:44.033: INFO: Container readiness ready: false, restart count 0 Nov 27 23:28:44.661: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:28:44.661: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:44.805: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 17932 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8085":"ip-172-20-50-151.sa-east-1.compute.internal","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:28:17 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:28:28 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2b5f598c-6eab-11ed-8b02-be523c12f2cd kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8085^2b5f598c-6eab-11ed-8b02-be523c12f2cd,DevicePath:,},},Config:nil,},} Nov 27 23:28:44.806: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:44.954: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:45.102: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.102: INFO: Container coredns ready: true, restart count 0 Nov 27 23:28:45.102: INFO: netserver-2 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.102: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:45.102: INFO: inline-volume-tester2-69klq started at 2022-11-27 23:28:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.102: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:28:45.102: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:45.102: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:45.102: INFO: inline-volume-tester-sfkx4 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:45.102: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 27 23:28:45.102: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:45.102: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:45.102: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:28:14 +0000 UTC (0+7 container statuses recorded) Nov 27 23:28:45.102: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:45.102: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:45.607: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:28:45.607: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:45.759: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 15987 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:27:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:28:45.760: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:45.906: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:46.207: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:46.207: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:46.207: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:28:46.207: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:28:46.207: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:28:46.207: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:28:46.207: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:46.207: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:28:46.207: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:46.207: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:28:46.207: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:28:46.207: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:28:46.207: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:28:46.207: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:28:46.207: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:46.207: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:28:46.722: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:28:46.722: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:46.868: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 18120 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8564":"csi-mock-csi-mock-volumes-8564","ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:28:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:28:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:28:37 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:28:37 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-8564^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-8564^4,DevicePath:,},},Config:nil,},} Nov 27 23:28:46.868: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:47.015: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:47.171: INFO: netserver-3 started at 2022-11-27 23:28:17 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container webserver ready: true, restart count 0 Nov 27 23:28:47.172: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:47.172: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:28:47.172: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:28:47.172: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:28:47.172: INFO: test-webserver-48243c59-6a81-4fab-8553-ae5f710c7444 started at 2022-11-27 23:25:01 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container test-webserver ready: true, restart count 0 Nov 27 23:28:47.172: INFO: csi-mockplugin-0 started at 2022-11-27 23:28:24 +0000 UTC (0+3 container statuses recorded) Nov 27 23:28:47.172: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:28:47.172: INFO: Container driver-registrar ready: true, restart count 0 Nov 27 23:28:47.172: INFO: Container mock ready: true, restart count 0 Nov 27 23:28:47.172: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-2gspb started at 2022-11-27 23:28:45 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container agnhost-container ready: false, restart count 0 Nov 27 23:28:47.172: INFO: liveness-33e27314-ba60-45c8-9252-4faa7e734eaf started at 2022-11-27 23:25:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:28:47.172: INFO: adopt-release-trt5r started at 2022-11-27 23:28:15 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container c ready: true, restart count 0 Nov 27 23:28:47.172: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:28:47.172: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:28:47.172: INFO: busybox-8c290bb6-883a-4503-b7b8-48cf81b29b5a started at 2022-11-27 23:26:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container busybox ready: true, restart count 0 Nov 27 23:28:47.172: INFO: csi-mockplugin-attacher-0 started at 2022-11-27 23:28:24 +0000 UTC (0+1 container statuses recorded) Nov 27 23:28:47.172: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:28:47.694: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:28:47.694: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-8649" for this suite.
Find pod-30713434-a567-4be1-865e-086c7884b449 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sskips\sownership\schanges\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:32:58.153: Unexpected error: <*errors.errorString | 0xc0037d85e0>: { s: "pod \"pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf\" is not Running: timed out waiting for the condition", } pod "pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_20.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:27:56.385: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 27 23:27:57.110: INFO: Creating resource for dynamic PV Nov 27 23:27:57.110: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-4658-e2e-sckzqm8 �[1mSTEP�[0m: creating a claim Nov 27 23:27:57.261: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-4658 with fsgroup 1000 Nov 27 23:32:58.153: FAIL: Unexpected error: <*errors.errorString | 0xc0037d85e0>: { s: "pod \"pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf\" is not Running: timed out waiting for the condition", } pod "pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc002f27a20, 0xc001fdf0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000322600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000322600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000322600, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 27 23:32:58.297: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comp5m7l" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-4658". �[1mSTEP�[0m: Found 5 events. Nov 27 23:32:58.742: INFO: At 2022-11-27 23:27:57 +0000 UTC - event for ebs.csi.aws.comp5m7l: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:32:58.742: INFO: At 2022-11-27 23:27:57 +0000 UTC - event for ebs.csi.aws.comp5m7l: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:32:58.742: INFO: At 2022-11-27 23:27:57 +0000 UTC - event for ebs.csi.aws.comp5m7l: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-4658/ebs.csi.aws.comp5m7l" Nov 27 23:32:58.742: INFO: At 2022-11-27 23:28:07 +0000 UTC - event for ebs.csi.aws.comp5m7l: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-4658-e2e-sckzqm8": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:32:58.742: INFO: At 2022-11-27 23:32:58 +0000 UTC - event for pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-4658/ebs.csi.aws.comp5m7l" Nov 27 23:32:58.888: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:32:58.888: INFO: pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:32:58 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "fsgroupchangepolicy-4658/ebs.csi.aws.comp5m7l"}] Nov 27 23:32:58.888: INFO: Nov 27 23:32:59.033: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:32:59.178: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 25340 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-27 23:31:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-27 23:31:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:31:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:31:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:31:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:31:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:32:59.178: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:32:59.328: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-rdl8s started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-hggg6 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-m94cq started at 2022-11-27 23:31:05 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:32:59.629: INFO: service-headless-5lhgs started at <nil> (0+0 container statuses recorded) Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-bc6zf started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: false, restart count 0 Nov 27 23:32:59.629: INFO: pod-subpath-test-secret-4l2q started at 2022-11-27 23:32:46 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container test-container-subpath-secret-4l2q ready: true, restart count 0 Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-gg968 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:32:59.629: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:32:59.629: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-5xsgv started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:32:59.629: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:32:59.629: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:32:59.629: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:32:59.629: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:32:59.629: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-9s2k2 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:32:59.629: INFO: httpd started at 2022-11-27 23:32:54 +0000 UTC (0+1 container statuses recorded) Nov 27 23:32:59.629: INFO: Container httpd ready: false, restart count 0 Nov 27 23:33:00.234: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:33:00.234: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:00.379: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 24672 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:30:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-27 23:31:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:30:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:33:00.379: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:00.526: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:00.676: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-nxrtj started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:00.677: INFO: boom-server started at 2022-11-27 23:32:34 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container boom-server ready: false, restart count 0 Nov 27 23:33:00.677: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:00.677: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:00.677: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:00.677: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:00.677: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-mqh2t started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:00.677: INFO: pod-subpath-test-preprovisionedpv-9q95 started at 2022-11-27 23:32:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Init container init-volume-preprovisionedpv-9q95 ready: false, restart count 0 Nov 27 23:33:00.677: INFO: Container test-container-subpath-preprovisionedpv-9q95 ready: false, restart count 0 Nov 27 23:33:00.677: INFO: pod-submit-remove-1faa18e5-11d1-4352-8e0b-e414f44a3dca started at 2022-11-27 23:32:22 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container agnhost-container ready: false, restart count 0 Nov 27 23:33:00.677: INFO: pod-client started at 2022-11-27 23:32:26 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container pod-client ready: false, restart count 0 Nov 27 23:33:00.677: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-qdkz4 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:00.677: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:00.677: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:00.677: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:33:00.677: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-hhtqn started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:00.677: INFO: hostexec-ip-172-20-34-252.sa-east-1.compute.internal-5ddsk started at 2022-11-27 23:31:03 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:00.677: INFO: hostexec-ip-172-20-34-252.sa-east-1.compute.internal-wr97f started at 2022-11-27 23:32:42 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:00.677: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-zt2zx started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:00.677: INFO: service-headless-4cghp started at <nil> (0+0 container statuses recorded) Nov 27 23:33:00.677: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-mx5d5 started at 2022-11-27 23:31:05 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:00.677: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container coredns ready: true, restart count 0 Nov 27 23:33:00.677: INFO: pod-aa1078fd-af70-47cd-951b-a000284209da started at 2022-11-27 23:31:27 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:00.677: INFO: Container write-pod ready: false, restart count 0 Nov 27 23:33:01.353: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:33:01.353: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:01.498: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 26365 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5756":"ip-172-20-50-151.sa-east-1.compute.internal","csi-hostpath-provisioning-5434":"ip-172-20-50-151.sa-east-1.compute.internal","ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-27 23:28:17 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {e2e.test Update v1 2022-11-27 23:31:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:38 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:38 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:38 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:32:38 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd kubernetes.io/csi/csi-hostpath-provisioning-5434^c0a1b9e8-6eab-11ed-996b-7e35453fb5d1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5434^c0a1b9e8-6eab-11ed-996b-7e35453fb5d1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd,DevicePath:,},},Config:nil,},} Nov 27 23:33:01.499: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:01.646: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:01.795: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:01.795: INFO: Container coredns ready: true, restart count 0 Nov 27 23:33:01.795: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:31:28 +0000 UTC (0+7 container statuses recorded) Nov 27 23:33:01.795: INFO: Container csi-attacher ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container csi-provisioner ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container csi-resizer ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container hostpath ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container liveness-probe ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 27 23:33:01.795: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:01.795: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:01.795: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-s5fs5 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:01.795: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:01.795: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:31:19 +0000 UTC (0+7 container statuses recorded) Nov 27 23:33:01.795: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:01.795: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:01.795: INFO: pod-subpath-test-dynamicpv-lxgn started at 2022-11-27 23:32:29 +0000 UTC (2+2 container statuses recorded) Nov 27 23:33:01.795: INFO: Init container init-volume-dynamicpv-lxgn ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Init container test-init-subpath-dynamicpv-lxgn ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container test-container-subpath-dynamicpv-lxgn ready: false, restart count 0 Nov 27 23:33:01.795: INFO: Container test-container-volume-dynamicpv-lxgn ready: false, restart count 0 Nov 27 23:33:01.796: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-pxwvh started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:01.796: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:01.796: INFO: pod-subpath-test-preprovisionedpv-8fsh started at 2022-11-27 23:31:25 +0000 UTC (2+1 container statuses recorded) Nov 27 23:33:01.796: INFO: Init container init-volume-preprovisionedpv-8fsh ready: false, restart count 0 Nov 27 23:33:01.796: INFO: Init container test-init-volume-preprovisionedpv-8fsh ready: false, restart count 0 Nov 27 23:33:01.796: INFO: Container test-container-subpath-preprovisionedpv-8fsh ready: false, restart count 0 Nov 27 23:33:01.796: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-bs2bt started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:01.796: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:01.796: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:01.796: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:01.796: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:01.796: INFO: hostexec-ip-172-20-50-151.sa-east-1.compute.internal-66zl2 started at 2022-11-27 23:31:06 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:01.796: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:02.388: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:33:02.388: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:02.533: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 25553 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42913173,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441814,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955310,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:33:02.534: INFO: Logging kubelet events for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:02.681: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:02.832: INFO: etcd-manager-main-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:33:02.832: INFO: kube-apiserver-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+2 container statuses recorded) Nov 27 23:33:02.832: INFO: Container healthcheck ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container kube-apiserver ready: true, restart count 1 Nov 27 23:33:02.832: INFO: kube-scheduler-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container kube-scheduler ready: true, restart count 0 Nov 27 23:33:02.832: INFO: ebs-csi-controller-764b64d679-flscb started at 2022-11-27 23:15:28 +0000 UTC (0+5 container statuses recorded) Nov 27 23:33:02.832: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:02.832: INFO: kops-controller-ftfzf started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container kops-controller ready: true, restart count 0 Nov 27 23:33:02.832: INFO: dns-controller-5fb778d9d5-sjmcb started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container dns-controller ready: true, restart count 0 Nov 27 23:33:02.832: INFO: etcd-manager-events-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container etcd-manager ready: true, restart count 0 Nov 27 23:33:02.832: INFO: kube-controller-manager-ip-172-20-50-174.sa-east-1.compute.internal started at 2022-11-27 23:14:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 27 23:33:02.832: INFO: cilium-rvvg4 started at 2022-11-27 23:15:25 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:02.832: INFO: ebs-csi-node-8q25m started at 2022-11-27 23:15:25 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:02.832: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:02.832: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:02.832: INFO: cilium-operator-587dd66f87-ckp6s started at 2022-11-27 23:15:28 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:02.832: INFO: Container cilium-operator ready: true, restart count 0 Nov 27 23:33:03.349: INFO: Latency metrics for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:33:03.349: INFO: Logging node info for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:03.493: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-210.sa-east-1.compute.internal ec2563a6-15f9-4d0d-bb28-738f22abb13f 24162 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-210.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-58-210.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06ac0fab5c679fa04"} io.cilium.network.ipv4-cilium-host:172.20.59.184 io.cilium.network.ipv4-health-ip:172.20.58.8 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:30:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-27 23:31:03 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06ac0fab5c679fa04,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:30:17 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.210,},NodeAddress{Type:ExternalIP,Address:18.228.195.108,},NodeAddress{Type:Hostname,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-210.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-195-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c510c0bb6d0174959f844de25eb40,SystemUUID:ec2c510c-0bb6-d017-4959-f844de25eb40,BootID:9d82dd36-8d8d-49b7-bbf5-a5b113b00e04,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:33:03.494: INFO: Logging kubelet events for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:03.641: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:03.789: INFO: local-injector started at 2022-11-27 23:32:56 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Container local-injector ready: false, restart count 0 Nov 27 23:33:03.789: INFO: ebs-csi-node-ppcrr started at 2022-11-27 23:16:56 +0000 UTC (0+3 container statuses recorded) Nov 27 23:33:03.789: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:33:03.789: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:33:03.789: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:33:03.789: INFO: hostexec-ip-172-20-58-210.sa-east-1.compute.internal-7mzz6 started at 2022-11-27 23:32:34 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:33:03.789: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-ghhr7 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:03.789: INFO: service-headless-pt5zc started at <nil> (0+0 container statuses recorded) Nov 27 23:33:03.789: INFO: cilium-kflj6 started at 2022-11-27 23:16:56 +0000 UTC (1+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:33:03.789: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:33:03.789: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-gjqx7 started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:03.789: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-6hc8g started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:03.789: INFO: cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a-vvzqh started at 2022-11-27 23:31:04 +0000 UTC (0+1 container statuses recorded) Nov 27 23:33:03.789: INFO: Container cleanup40-6a8244c3-ecfd-4a07-81e5-66baca703d6a ready: true, restart count 0 Nov 27 23:33:04.378: INFO: Latency metrics for node ip-172-20-58-210.sa-east-1.compute.internal Nov 27 23:33:04.378: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-4658" for this suite.
Find pod-bf1a93b3-25ad-4de7-b7c1-6bd174bbeabf mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\smount\soptions$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 27 23:35:15.062: Unexpected error: <*errors.errorString | 0xc003cea0f0>: { s: "pod \"pod-cfc47689-c54a-4b9a-8691-2872c7549cfe\" is not Running: timed out waiting for the condition", } pod "pod-cfc47689-c54a-4b9a-8691-2872c7549cfe" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 27 23:30:13.037: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should provision storage with mount options /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 27 23:30:13.759: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 27 23:30:13.759: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 27 23:30:13.759: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-7478-e2e-sc9srcz 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-7478 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7478-e2e-sc9srcz,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-7478 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7478-e2e-sc9srcz,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-7478 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7478-e2e-sc9srcz,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-7478-e2e-sc9srcz 4673aa31-6abb-4443-9dbd-ced3fff9bdc1 22551 0 2022-11-27 23:30:13 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-11-27 23:30:13 +0000 UTC FieldsV1 {"f:allowVolumeExpansion":{},"f:mountOptions":{},"f:parameters":{".":{},"f:encrypted":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:*Delete,MountOptions:[dirsync],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-m8s26 pvc- provisioning-7478 0588843e-f2d1-4711-9a5a-ba998f2c8e9e 22565 0 2022-11-27 23:30:14 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-11-27 23:30:14 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7478-e2e-sc9srcz,VolumeMode:*Filesystem,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} Nov 27 23:35:15.062: FAIL: Unexpected error: <*errors.errorString | 0xc003cea0f0>: { s: "pod \"pod-cfc47689-c54a-4b9a-8691-2872c7549cfe\" is not Running: timed out waiting for the condition", } pod "pod-cfc47689-c54a-4b9a-8691-2872c7549cfe" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning(0x7973fe8, 0xc0041d5e40, 0xc003e94850, 0xc00400e540, 0xc00400e700, 0xc004218780, 0x0, 0x0, 0x0, 0x0, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418 +0xe7d k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:198 +0x245 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000cb2000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000cb2000) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000cb2000, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 27 23:35:15.063: INFO: deleting claim "provisioning-7478"/"pvc-m8s26" Nov 27 23:35:15.210: INFO: deleting storage class provisioning-7478-e2e-sc9srcz [AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "provisioning-7478". �[1mSTEP�[0m: Found 5 events. Nov 27 23:35:15.506: INFO: At 2022-11-27 23:30:14 +0000 UTC - event for pvc-m8s26: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 27 23:35:15.506: INFO: At 2022-11-27 23:30:14 +0000 UTC - event for pvc-m8s26: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 27 23:35:15.506: INFO: At 2022-11-27 23:30:14 +0000 UTC - event for pvc-m8s26: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } Provisioning: External provisioner is provisioning volume for claim "provisioning-7478/pvc-m8s26" Nov 27 23:35:15.506: INFO: At 2022-11-27 23:30:24 +0000 UTC - event for pvc-m8s26: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "provisioning-7478-e2e-sc9srcz": rpc error: code = Internal desc = Could not create volume "pvc-0588843e-f2d1-4711-9a5a-ba998f2c8e9e": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 27 23:35:15.506: INFO: At 2022-11-27 23:30:35 +0000 UTC - event for pvc-m8s26: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-flscb_839ff2ea-4910-4e0c-8f6a-f77a8cc0241c } ProvisioningFailed: failed to provision volume with StorageClass "provisioning-7478-e2e-sc9srcz": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 27 23:35:15.649: INFO: POD NODE PHASE GRACE CONDITIONS Nov 27 23:35:15.650: INFO: pod-cfc47689-c54a-4b9a-8691-2872c7549cfe Pending [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2022-11-27 23:35:15 +0000 UTC SchedulerError running PreBind plugin "VolumeBinding": binding volumes: failed to check provisioning pvc: could not find v1.PersistentVolumeClaim "provisioning-7478/pvc-m8s26"}] Nov 27 23:35:15.650: INFO: Nov 27 23:35:15.795: INFO: Logging node info for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:15.939: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-127.sa-east-1.compute.internal 17649054-a7e6-4c50-8129-e6a01563fdfa 30106 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-127.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-33-127.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05b0a5a64b16c8008"} io.cilium.network.ipv4-cilium-host:172.20.36.79 io.cilium.network.ipv4-health-ip:172.20.39.207 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:34:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-05b0a5a64b16c8008,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:26 +0000 UTC,LastTransitionTime:2022-11-27 23:17:26 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:34:57 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.127,},NodeAddress{Type:ExternalIP,Address:15.228.239.94,},NodeAddress{Type:Hostname,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-127.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-228-239-94.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d0a8096e86412c16049d644fdf3a,SystemUUID:ec21d0a8-096e-8641-2c16-049d644fdf3a,BootID:8219af7c-8fef-4452-9326-6851e284bcf1,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:15.940: INFO: Logging kubelet events for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:16.088: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:16.236: INFO: liveness-13d8a8f9-feb3-4cfb-8d05-b50b8a6dff39 started at 2022-11-27 23:34:21 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:16.236: INFO: Container agnhost-container ready: true, restart count 2 Nov 27 23:35:16.236: INFO: cilium-7rmwl started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:16.236: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:16.236: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:16.236: INFO: ebs-csi-node-zql72 started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:16.236: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:16.236: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:16.236: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:16.236: INFO: downwardapi-volume-fa73a6c8-87b5-4a10-88bb-5e423534db6a started at 2022-11-27 23:35:11 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:16.236: INFO: Container client-container ready: false, restart count 0 Nov 27 23:35:16.236: INFO: startup-ed1b587a-5f15-4fa7-b254-5826c007451d started at 2022-11-27 23:35:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:16.236: INFO: Container busybox ready: false, restart count 0 Nov 27 23:35:16.236: INFO: test-container-pod started at 2022-11-27 23:34:58 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:16.236: INFO: Container webserver ready: true, restart count 0 Nov 27 23:35:16.236: INFO: busybox-0a751258-a7fc-4812-9922-7655a15f522e started at 2022-11-27 23:35:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:16.236: INFO: Container busybox ready: true, restart count 0 Nov 27 23:35:16.780: INFO: Latency metrics for node ip-172-20-33-127.sa-east-1.compute.internal Nov 27 23:35:16.781: INFO: Logging node info for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:16.925: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-252.sa-east-1.compute.internal db425ead-0a82-4cf7-8d4f-6292e030920a 30359 0 2022-11-27 23:16:56 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-252.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-34-252.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volumemode-5799":"ip-172-20-34-252.sa-east-1.compute.internal","ebs.csi.aws.com":"i-037464738c55f9925"} io.cilium.network.ipv4-cilium-host:172.20.51.222 io.cilium.network.ipv4-health-ip:172.20.34.92 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:30:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-037464738c55f9925,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:16:56 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:27 +0000 UTC,LastTransitionTime:2022-11-27 23:17:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.252,},NodeAddress{Type:ExternalIP,Address:18.230.24.108,},NodeAddress{Type:Hostname,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-252.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-24-108.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21a4eccb04b2db59359ae0bd2c4d61,SystemUUID:ec21a4ec-cb04-b2db-5935-9ae0bd2c4d61,BootID:ccb14296-a701-4e5c-b29a-0b38f6d9b931,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:16.925: INFO: Logging kubelet events for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:17.072: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:17.220: INFO: cilium-7ll74 started at 2022-11-27 23:16:57 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:17.220: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:17.220: INFO: coredns-autoscaler-6658b4bf85-2r8ph started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:17.220: INFO: Container autoscaler ready: true, restart count 0 Nov 27 23:35:17.220: INFO: coredns-5fcc7b6498-zzw8h started at 2022-11-27 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:17.220: INFO: Container coredns ready: true, restart count 0 Nov 27 23:35:17.220: INFO: pod-secrets-326bd201-988e-4ecd-9c1b-d3fddb973706 started at 2022-11-27 23:34:30 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:17.220: INFO: Container creates-volume-test ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container dels-volume-test ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container upds-volume-test ready: true, restart count 0 Nov 27 23:35:17.220: INFO: csi-hostpathplugin-0 started at 2022-11-27 23:34:53 +0000 UTC (0+7 container statuses recorded) Nov 27 23:35:17.220: INFO: Container csi-attacher ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container csi-provisioner ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container csi-resizer ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container hostpath ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:17.220: INFO: ebs-csi-node-8qv2x started at 2022-11-27 23:16:57 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:17.220: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:17.220: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:17.220: INFO: hostexec-ip-172-20-34-252.sa-east-1.compute.internal-clbfs started at 2022-11-27 23:35:10 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:17.220: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:17.220: INFO: pause-pod-1 started at 2022-11-27 23:34:19 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:17.220: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:17.790: INFO: Latency metrics for node ip-172-20-34-252.sa-east-1.compute.internal Nov 27 23:35:17.790: INFO: Logging node info for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:17.935: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-151.sa-east-1.compute.internal 8b8b0739-c6ad-4185-b848-341abf868052 30281 0 2022-11-27 23:16:57 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-151.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:ip-172-20-50-151.sa-east-1.compute.internal topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c29c65de738cf719"} io.cilium.network.ipv4-cilium-host:172.20.49.97 io.cilium.network.ipv4-health-ip:172.20.49.29 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-27 23:16:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-27 23:17:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:17:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:28:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c29c65de738cf719,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:17:25 +0000 UTC,LastTransitionTime:2022-11-27 23:17:25 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:16:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:33:58 +0000 UTC,LastTransitionTime:2022-11-27 23:17:17 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.151,},NodeAddress{Type:ExternalIP,Address:18.231.64.155,},NodeAddress{Type:Hostname,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-151.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-231-64-155.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec231ef10482b09f400ab9421bf60e99,SystemUUID:ec231ef1-0482-b09f-400a-b9421bf60e99,BootID:01eeb1df-1dca-4442-99e3-2ee9c37ed993,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8085^2f5bd7ac-6eab-11ed-8b02-be523c12f2cd],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 27 23:35:17.935: INFO: Logging kubelet events for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:18.084: INFO: Logging pods the kubelet thinks is on node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:18.232: INFO: hostexec-ip-172-20-50-151.sa-east-1.compute.internal-fz8j6 started at 2022-11-27 23:35:05 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:18.232: INFO: ebs-csi-node-v7nmj started at 2022-11-27 23:16:58 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:18.232: INFO: Container ebs-plugin ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container liveness-probe ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 27 23:35:18.232: INFO: webserver-pod started at 2022-11-27 23:34:14 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Container agnhost ready: false, restart count 0 Nov 27 23:35:18.232: INFO: pod-subpath-test-preprovisionedpv-ccj4 started at 2022-11-27 23:35:11 +0000 UTC (2+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Init container init-volume-preprovisionedpv-ccj4 ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Init container test-init-volume-preprovisionedpv-ccj4 ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container test-container-subpath-preprovisionedpv-ccj4 ready: false, restart count 0 Nov 27 23:35:18.232: INFO: dns-test-cd95b407-6cea-45d5-8c93-153bde0e2337 started at 2022-11-27 23:34:54 +0000 UTC (0+3 container statuses recorded) Nov 27 23:35:18.232: INFO: Container jessie-querier ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container querier ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container webserver ready: true, restart count 0 Nov 27 23:35:18.232: INFO: cilium-bncbq started at 2022-11-27 23:16:58 +0000 UTC (1+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container cilium-agent ready: true, restart count 0 Nov 27 23:35:18.232: INFO: pod-subpath-test-preprovisionedpv-rsjp started at 2022-11-27 23:35:12 +0000 UTC (2+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Init container init-volume-preprovisionedpv-rsjp ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Init container test-init-volume-preprovisionedpv-rsjp ready: true, restart count 0 Nov 27 23:35:18.232: INFO: Container test-container-subpath-preprovisionedpv-rsjp ready: false, restart count 0 Nov 27 23:35:18.232: INFO: hostexec-ip-172-20-50-151.sa-east-1.compute.internal-7482n started at 2022-11-27 23:35:00 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Container agnhost-container ready: true, restart count 0 Nov 27 23:35:18.232: INFO: coredns-5fcc7b6498-n62sn started at 2022-11-27 23:17:49 +0000 UTC (0+1 container statuses recorded) Nov 27 23:35:18.232: INFO: Container coredns ready: true, restart count 0 Nov 27 23:35:20.633: INFO: Latency metrics for node ip-172-20-50-151.sa-east-1.compute.internal Nov 27 23:35:20.633: INFO: Logging node info for node ip-172-20-50-174.sa-east-1.compute.internal Nov 27 23:35:20.777: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-50-174.sa-east-1.compute.internal fd25ee65-df53-4264-a6b5-1baf754f4ed3 25553 0 2022-11-27 23:15:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-50-174.sa-east-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07f0376dc13e2d817"} io.cilium.network.ipv4-cilium-host:172.20.57.15 io.cilium.network.ipv4-health-ip:172.20.48.57 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-27 23:15:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-27 23:15:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-27 23:15:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-27 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-27 23:16:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-27 23:16:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-27 23:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-07f0376dc13e2d817,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-27 23:16:29 +0000 UTC,LastTransitionTime:2022-11-27 23:16:29 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:14:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-27 23:32:21 +0000 UTC,LastTransitionTime:2022-11-27 23:15:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.50.174,},NodeAddress{Type:ExternalIP,Address:18.228.235.250,},NodeAddress{Type:Hostname,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-50-174.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-250.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24b4f86c0a766b4de71733a9ccd932,SystemUUID:ec24b4f8-6c0a-766b-4de7-1733a9ccd932,BootID:123f10b7-53b8-464f-919c-eaaa7b3025fe,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,}