Result | FAILURE |
Tests | 44 failed / 722 succeeded |
Started | |
Elapsed | 47m10s |
Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 29 23:31:55.750: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc002b62ac0>: { s: "pod \"pod-b76105c9-da70-4051-ab05-298ebbf7e847\" is not Running: timed out waiting for the condition", } pod "pod-b76105c9-da70-4051-ab05-298ebbf7e847" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_10.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:26:55.394: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 29 23:26:55.539: INFO: Creating resource for dynamic PV Nov 29 23:26:55.539: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-7558-e2e-sclbstw �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 29 23:31:55.750: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc002b62ac0>: { s: "pod \"pod-b76105c9-da70-4051-ab05-298ebbf7e847\" is not Running: timed out waiting for the condition", } pod "pod-b76105c9-da70-4051-ab05-298ebbf7e847" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000227b00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000227b00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000227b00, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 29 23:31:55.750: INFO: Deleting pod "pod-b76105c9-da70-4051-ab05-298ebbf7e847" in namespace "volume-expand-7558" Nov 29 23:31:55.782: INFO: Wait up to 5m0s for pod "pod-b76105c9-da70-4051-ab05-298ebbf7e847" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 29 23:31:55.811: INFO: Deleting pod "pod-b76105c9-da70-4051-ab05-298ebbf7e847" in namespace "volume-expand-7558" �[1mSTEP�[0m: Deleting pvc Nov 29 23:31:55.870: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comxtc5b" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-7558". �[1mSTEP�[0m: Found 4 events. Nov 29 23:31:55.960: INFO: At 2022-11-29 23:26:55 +0000 UTC - event for ebs.csi.aws.comxtc5b: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:31:55.960: INFO: At 2022-11-29 23:26:55 +0000 UTC - event for ebs.csi.aws.comxtc5b: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:31:55.960: INFO: At 2022-11-29 23:26:55 +0000 UTC - event for ebs.csi.aws.comxtc5b: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-7558/ebs.csi.aws.comxtc5b" Nov 29 23:31:55.960: INFO: At 2022-11-29 23:27:05 +0000 UTC - event for ebs.csi.aws.comxtc5b: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-7558-e2e-sclbstw": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:31:55.989: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:31:55.989: INFO: Nov 29 23:31:56.019: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:56.053: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 26881 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4453":"csi-mock-csi-mock-volumes-4453","ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-29 23:29:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:56.054: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:56.085: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:56.123: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:56.123: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:56.123: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:56.123: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:56.123: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:56.123: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:56.123: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:56.123: INFO: hostexec-ip-172-20-34-50.us-east-2.compute.internal-9ckvr started at 2022-11-29 23:31:53 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.123: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:31:56.123: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.123: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:31:56.123: INFO: csi-mockplugin-0 started at 2022-11-29 23:31:44 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:56.123: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:31:56.123: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:31:56.123: INFO: Container mock ready: true, restart count 0 Nov 29 23:31:56.123: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.123: INFO: Container coredns ready: true, restart count 0 Nov 29 23:31:56.123: INFO: startup-script started at 2022-11-29 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.123: INFO: Container startup-script ready: true, restart count 0 Nov 29 23:31:56.277: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:56.277: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:56.307: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 20634 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:56.307: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:56.339: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:56.375: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:31:56.376: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:31:56.376: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:31:56.376: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:31:56.376: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:56.376: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:56.376: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:56.376: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:31:56.376: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:31:56.376: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:56.376: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:31:56.376: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:31:56.376: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:31:56.376: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:31:56.376: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.376: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:31:56.546: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:56.546: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:56.578: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 26823 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:28:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:56.578: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:56.611: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:56.643: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:56.643: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:56.643: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:56.643: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:56.643: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:56.643: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:56.643: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:56.643: INFO: rs-gzl25 started at 2022-11-29 23:31:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.643: INFO: Container donothing ready: false, restart count 0 Nov 29 23:31:56.643: INFO: pod-subpath-test-configmap-dn2z started at 2022-11-29 23:31:49 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.643: INFO: Container test-container-subpath-configmap-dn2z ready: true, restart count 0 Nov 29 23:31:56.643: INFO: rs-bnfw9 started at 2022-11-29 23:31:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.643: INFO: Container donothing ready: false, restart count 0 Nov 29 23:31:56.643: INFO: pod-ephm-test-projected-vxrn started at 2022-11-29 23:30:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.643: INFO: Container test-container-subpath-projected-vxrn ready: false, restart count 0 Nov 29 23:31:56.799: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:56.799: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:56.828: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 24907 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:30:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:56.829: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:56.860: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:56.897: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.897: INFO: Container coredns ready: true, restart count 0 Nov 29 23:31:56.897: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:56.897: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:56.897: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:56.897: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:56.897: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:56.897: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:56.897: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:56.897: INFO: pod-secrets-458971a2-cfa1-4a29-a534-a834593918ec started at 2022-11-29 23:30:25 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:56.897: INFO: Container creates-volume-test ready: true, restart count 0 Nov 29 23:31:56.897: INFO: Container dels-volume-test ready: true, restart count 0 Nov 29 23:31:56.897: INFO: Container upds-volume-test ready: true, restart count 0 Nov 29 23:31:56.897: INFO: rs-9bpbh started at 2022-11-29 23:30:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.897: INFO: Container donothing ready: false, restart count 0 Nov 29 23:31:56.897: INFO: rs-q9n5p started at 2022-11-29 23:31:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:56.897: INFO: Container donothing ready: true, restart count 0 Nov 29 23:31:57.048: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:57.049: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:57.078: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 27143 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-143":"ip-172-20-58-90.us-east-2.compute.internal","ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:24:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-11-29 23:31:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-143^02189298-703e-11ed-b86f-7e98746e0dd1,DevicePath:,},},Config:nil,},} Nov 29 23:31:57.078: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:57.114: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:57.146: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:31:52 +0000 UTC (0+7 container statuses recorded) Nov 29 23:31:57.146: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:57.146: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:57.146: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:57.146: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:57.146: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:57.146: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:57.146: INFO: boom-server started at 2022-11-29 23:30:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:57.146: INFO: Container boom-server ready: true, restart count 0 Nov 29 23:31:57.146: INFO: test-webserver-8f294c7e-7a26-4a12-a2d9-28c256c07264 started at 2022-11-29 23:28:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:57.146: INFO: Container test-webserver ready: true, restart count 0 Nov 29 23:31:57.146: INFO: pod-subpath-test-dynamicpv-jsbm started at 2022-11-29 23:31:56 +0000 UTC (2+2 container statuses recorded) Nov 29 23:31:57.146: INFO: Init container init-volume-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:31:57.146: INFO: Init container test-init-subpath-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:31:57.146: INFO: Container test-container-subpath-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:31:57.146: INFO: Container test-container-volume-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:31:57.341: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:57.341: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-7558" for this suite.
Find pod-b76105c9-da70-4051-ab05-298ebbf7e847 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 29 23:45:42.131: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc0024c0650>: { s: "pod \"pod-dceb0bab-800c-4258-abcd-977d708d01de\" is not Running: timed out waiting for the condition", } pod "pod-dceb0bab-800c-4258-abcd-977d708d01de" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_18.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:40:41.764: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 29 23:40:41.913: INFO: Creating resource for dynamic PV Nov 29 23:40:41.913: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-3272-e2e-schxfqp �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 29 23:45:42.131: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc0024c0650>: { s: "pod \"pod-dceb0bab-800c-4258-abcd-977d708d01de\" is not Running: timed out waiting for the condition", } pod "pod-dceb0bab-800c-4258-abcd-977d708d01de" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000886480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000886480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000886480, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 29 23:45:42.132: INFO: Deleting pod "pod-dceb0bab-800c-4258-abcd-977d708d01de" in namespace "volume-expand-3272" Nov 29 23:45:42.165: INFO: Wait up to 5m0s for pod "pod-dceb0bab-800c-4258-abcd-977d708d01de" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 29 23:45:42.195: INFO: Deleting pod "pod-dceb0bab-800c-4258-abcd-977d708d01de" in namespace "volume-expand-3272" �[1mSTEP�[0m: Deleting pvc Nov 29 23:45:42.254: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comqscdl" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-3272". �[1mSTEP�[0m: Found 4 events. Nov 29 23:45:42.346: INFO: At 2022-11-29 23:40:41 +0000 UTC - event for ebs.csi.aws.comqscdl: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:45:42.346: INFO: At 2022-11-29 23:40:42 +0000 UTC - event for ebs.csi.aws.comqscdl: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:45:42.346: INFO: At 2022-11-29 23:40:42 +0000 UTC - event for ebs.csi.aws.comqscdl: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-3272/ebs.csi.aws.comqscdl" Nov 29 23:45:42.346: INFO: At 2022-11-29 23:40:52 +0000 UTC - event for ebs.csi.aws.comqscdl: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-3272-e2e-schxfqp": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:45:42.375: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:45:42.375: INFO: Nov 29 23:45:42.406: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:42.436: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 46970 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:38:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:42.436: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:42.470: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:42.502: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.503: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:45:42.503: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.503: INFO: Container coredns ready: true, restart count 0 Nov 29 23:45:42.503: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:42.503: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:42.503: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:42.503: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:42.503: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:42.503: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:42.503: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:42.655: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:42.655: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:42.685: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 46590 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:42.685: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:42.718: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:42.753: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:42.753: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:45:42.753: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:45:42.753: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:42.753: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:45:42.753: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:45:42.753: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:45:42.753: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:45:42.753: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:45:42.753: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:45:42.753: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:45:42.753: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:42.753: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:45:42.753: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:42.753: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:42.753: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:42.905: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:42.905: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:42.935: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 47319 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:42.935: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:42.968: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:43.000: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:43.000: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:43.000: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:43.000: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:43.000: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:43.000: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:43.000: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:43.172: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:43.172: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:43.202: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 47237 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:43.203: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:43.236: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:43.269: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:43.269: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:43.269: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:43.269: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:43.269: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:43.269: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:43.269: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:43.269: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:43.269: INFO: Container coredns ready: true, restart count 0 Nov 29 23:45:43.455: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:43.455: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:43.485: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 46968 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:36:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:43.485: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:43.518: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:43.549: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:43.549: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:43.549: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:43.550: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:43.550: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:43.550: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:43.550: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:43.698: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:43.698: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-3272" for this suite.
Find pod-dceb0bab-800c-4258-abcd-977d708d01de mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 29 23:30:27.369: Unexpected error: <*errors.errorString | 0xc0002ba280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385from junit_14.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:25:26.993: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volumemode �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should not mount / map unused volumes in a pod [LinuxOnly] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 29 23:25:27.139: INFO: Creating resource for dynamic PV Nov 29 23:25:27.139: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volumemode-673-e2e-sczkqsl �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating pod Nov 29 23:30:27.368: FAIL: Unexpected error: <*errors.errorString | 0xc0002ba280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385 +0x6c5 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000187500) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000187500) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000187500, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 29 23:30:27.369: INFO: Deleting pod "pod-49a293b4-4346-41bc-b095-4d9d4e9efb97" in namespace "volumemode-673" Nov 29 23:30:27.405: INFO: Wait up to 5m0s for pod "pod-49a293b4-4346-41bc-b095-4d9d4e9efb97" to be fully deleted �[1mSTEP�[0m: Deleting pvc Nov 29 23:30:27.464: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comcljz5" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volumemode-673". �[1mSTEP�[0m: Found 5 events. Nov 29 23:30:27.563: INFO: At 2022-11-29 23:25:27 +0000 UTC - event for ebs.csi.aws.comcljz5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:30:27.563: INFO: At 2022-11-29 23:25:27 +0000 UTC - event for ebs.csi.aws.comcljz5: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:30:27.563: INFO: At 2022-11-29 23:25:27 +0000 UTC - event for ebs.csi.aws.comcljz5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "volumemode-673/ebs.csi.aws.comcljz5" Nov 29 23:30:27.563: INFO: At 2022-11-29 23:25:37 +0000 UTC - event for ebs.csi.aws.comcljz5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-673-e2e-sczkqsl": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:30:27.563: INFO: At 2022-11-29 23:28:54 +0000 UTC - event for ebs.csi.aws.comcljz5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-673-e2e-sczkqsl": rpc error: code = Internal desc = Could not create volume "pvc-fbfdc024-354a-4ea9-aa08-da8c77e29583": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 29 23:30:27.594: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:30:27.594: INFO: Nov 29 23:30:27.632: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:30:27.662: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 24597 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-29 23:29:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:29:56 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:29:56 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:29:56 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:29:56 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:30:27.663: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:30:27.696: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:30:27.737: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:30:27.737: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:30:27.737: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:30:27.737: INFO: webserver-pod started at 2022-11-29 23:25:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.737: INFO: Container agnhost ready: false, restart count 0 Nov 29 23:30:27.737: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:30:27.737: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:30:27.737: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:27.737: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:30:27.737: INFO: netserver-0 started at 2022-11-29 23:30:13 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.738: INFO: Container webserver ready: false, restart count 0 Nov 29 23:30:27.738: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.738: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:30:27.738: INFO: agnhost started at 2022-11-29 23:30:26 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.738: INFO: Container agnhost ready: false, restart count 0 Nov 29 23:30:27.738: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.738: INFO: Container coredns ready: true, restart count 0 Nov 29 23:30:27.738: INFO: service-proxy-disabled-8l9j9 started at 2022-11-29 23:29:53 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.738: INFO: Container service-proxy-disabled ready: true, restart count 0 Nov 29 23:30:27.738: INFO: test-rolling-update-with-lb-5f85c5c5f4-zw96r started at 2022-11-29 23:30:03 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:27.738: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:30:27.931: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:30:27.931: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:30:27.965: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 20634 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:30:27.966: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:30:27.999: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:30:28.039: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:30:28.039: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:30:28.039: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:30:28.039: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:30:28.039: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:28.039: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:30:28.039: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:30:28.039: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:30:28.039: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:30:28.039: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:28.039: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:30:28.039: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:30:28.039: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:30:28.039: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:30:28.039: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.039: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:30:28.260: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:30:28.261: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:30:28.291: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 25025 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3872":"ip-172-20-40-36.us-east-2.compute.internal","ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:28:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:30:28.292: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:30:28.328: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:30:28.383: INFO: test-rolling-update-with-lb-5f85c5c5f4-c59j4 started at 2022-11-29 23:29:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:30:28.384: INFO: httpd-deployment-8584777d8-n45td started at 2022-11-29 23:28:29 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container httpd ready: false, restart count 0 Nov 29 23:30:28.384: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:30:28.384: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:30:28.384: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:30:28.384: INFO: inline-volume-tester2-wvl5k started at 2022-11-29 23:30:27 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 29 23:30:28.384: INFO: service-proxy-toggled-6bb8n started at 2022-11-29 23:29:59 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container service-proxy-toggled ready: true, restart count 0 Nov 29 23:30:28.384: INFO: netserver-1 started at 2022-11-29 23:30:13 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container webserver ready: false, restart count 0 Nov 29 23:30:28.384: INFO: inline-volume-tester-5swrh started at 2022-11-29 23:30:21 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 29 23:30:28.384: INFO: pod-ephm-test-projected-vxrn started at 2022-11-29 23:30:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container test-container-subpath-projected-vxrn ready: false, restart count 0 Nov 29 23:30:28.384: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:30:21 +0000 UTC (0+7 container statuses recorded) Nov 29 23:30:28.384: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:28.384: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:30:28.384: INFO: service-proxy-toggled-6jf6s started at 2022-11-29 23:30:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container service-proxy-toggled ready: true, restart count 0 Nov 29 23:30:28.384: INFO: service-proxy-disabled-nwgmn started at 2022-11-29 23:29:53 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.384: INFO: Container service-proxy-disabled ready: true, restart count 0 Nov 29 23:30:28.701: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:30:28.701: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:30:28.748: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 24907 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:30:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:30:28.749: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:30:28.797: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:30:28.832: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.832: INFO: Container coredns ready: true, restart count 0 Nov 29 23:30:28.832: INFO: verify-service-up-host-exec-pod started at 2022-11-29 23:30:21 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.832: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:30:28.832: INFO: verify-service-up-exec-pod-65z9v started at 2022-11-29 23:30:23 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.832: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:30:28.832: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:30:28.832: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:30:28.832: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:28.832: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:30:28.832: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:30:28.832: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:30:28.832: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:30:28.832: INFO: netserver-2 started at 2022-11-29 23:30:13 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.832: INFO: Container webserver ready: false, restart count 0 Nov 29 23:30:28.832: INFO: pod-secrets-458971a2-cfa1-4a29-a534-a834593918ec started at 2022-11-29 23:30:25 +0000 UTC (0+3 container statuses recorded) Nov 29 23:30:28.832: INFO: Container creates-volume-test ready: true, restart count 0 Nov 29 23:30:28.832: INFO: Container dels-volume-test ready: true, restart count 0 Nov 29 23:30:28.832: INFO: Container upds-volume-test ready: true, restart count 0 Nov 29 23:30:28.832: INFO: rs-9bpbh started at 2022-11-29 23:30:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:28.832: INFO: Container donothing ready: false, restart count 0 Nov 29 23:30:29.054: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:30:29.054: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:30:29.083: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 20534 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:24:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:30:29.083: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:30:29.120: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:30:29.161: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:30:29.161: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:30:29.161: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:30:29.161: INFO: service-proxy-disabled-n8pv6 started at 2022-11-29 23:29:53 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:29.161: INFO: Container service-proxy-disabled ready: true, restart count 0 Nov 29 23:30:29.161: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:30:29.161: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:30:29.161: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:30:29.161: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:30:29.161: INFO: service-proxy-toggled-lbjjs started at 2022-11-29 23:30:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:29.161: INFO: Container service-proxy-toggled ready: true, restart count 0 Nov 29 23:30:29.161: INFO: test-webserver-8f294c7e-7a26-4a12-a2d9-28c256c07264 started at 2022-11-29 23:28:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:29.161: INFO: Container test-webserver ready: true, restart count 0 Nov 29 23:30:29.161: INFO: test-rolling-update-with-lb-5f85c5c5f4-jr5rq started at 2022-11-29 23:30:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:29.161: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:30:29.161: INFO: netserver-3 started at 2022-11-29 23:30:13 +0000 UTC (0+1 container statuses recorded) Nov 29 23:30:29.161: INFO: Container webserver ready: false, restart count 0 Nov 29 23:30:29.320: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:30:29.320: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volumemode-673" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumes\sshould\sstore\sdata$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 29 23:25:47.510: Failed to create injector pod: timed out waiting for the condition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186from junit_08.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:20:45.797: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should store data /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 29 23:20:47.227: INFO: Creating resource for dynamic PV Nov 29 23:20:47.227: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volume-1069-e2e-scw5chj �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: starting external-injector �[1mSTEP�[0m: Deleting pod external-injector in namespace volume-1069 Nov 29 23:25:47.481: INFO: Waiting for pod external-injector to disappear Nov 29 23:25:47.510: INFO: Pod external-injector no longer exists Nov 29 23:25:47.510: FAIL: Failed to create injector pod: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumesTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186 +0x3ff k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000c3b380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000c3b380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000c3b380, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: cleaning the environment after external �[1mSTEP�[0m: Deleting pvc Nov 29 23:25:47.540: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comwtxfm" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-1069". �[1mSTEP�[0m: Found 4 events. Nov 29 23:25:47.639: INFO: At 2022-11-29 23:20:47 +0000 UTC - event for ebs.csi.aws.comwtxfm: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:25:47.639: INFO: At 2022-11-29 23:20:47 +0000 UTC - event for ebs.csi.aws.comwtxfm: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:25:47.639: INFO: At 2022-11-29 23:20:47 +0000 UTC - event for ebs.csi.aws.comwtxfm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "volume-1069/ebs.csi.aws.comwtxfm" Nov 29 23:25:47.639: INFO: At 2022-11-29 23:20:57 +0000 UTC - event for ebs.csi.aws.comwtxfm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volume-1069-e2e-scw5chj": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:25:47.668: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:25:47.668: INFO: Nov 29 23:25:47.725: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:25:47.755: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 16675 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4312":"csi-mock-csi-mock-volumes-4312","ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:25:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:25:47.756: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:25:47.793: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:25:47.872: INFO: webserver-57d64b6f67-5wwll started at 2022-11-29 23:25:46 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container httpd ready: false, restart count 0 Nov 29 23:25:47.873: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:25:47.873: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:25:47.873: INFO: csi-mockplugin-0 started at 2022-11-29 23:25:14 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:47.873: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:25:47.873: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:25:47.873: INFO: Container mock ready: true, restart count 0 Nov 29 23:25:47.873: INFO: webserver-pod started at 2022-11-29 23:25:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:25:47.873: INFO: pod-subpath-test-secret-t2xn started at 2022-11-29 23:25:39 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container test-container-subpath-secret-t2xn ready: true, restart count 0 Nov 29 23:25:47.873: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:47.873: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:25:47.873: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:47.873: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:47.873: INFO: webserver-57d64b6f67-znszk started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container httpd ready: true, restart count 0 Nov 29 23:25:47.873: INFO: csi-mockplugin-attacher-0 started at 2022-11-29 23:25:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:25:47.873: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:25:47.873: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:47.873: INFO: Container coredns ready: true, restart count 0 Nov 29 23:25:48.134: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:25:48.134: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:25:48.163: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 8195 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:25:48.163: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:25:48.194: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:25:48.231: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:48.231: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:48.231: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:25:48.231: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:25:48.231: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:25:48.231: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:25:48.231: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:25:48.231: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:25:48.231: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:25:48.231: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:25:48.231: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:48.231: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:25:48.231: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:25:48.231: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:25:48.231: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.231: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:25:48.469: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:25:48.469: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:25:48.498: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 17295 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-7903":"ip-172-20-40-36.us-east-2.compute.internal","csi-hostpath-volume-expand-5986":"ip-172-20-40-36.us-east-2.compute.internal","ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-29 23:25:45 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:47 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:47 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:47 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:25:47 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-5986^253b9d37-703d-11ed-9c2c-02d38bb0a487],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5986^253b9d37-703d-11ed-9c2c-02d38bb0a487,DevicePath:,},},Config:nil,},} Nov 29 23:25:48.499: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:25:48.533: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:25:48.573: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:24:54 +0000 UTC (0+7 container statuses recorded) Nov 29 23:25:48.573: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:48.573: INFO: webserver-57d64b6f67-dhptm started at 2022-11-29 23:25:47 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.573: INFO: Container httpd ready: false, restart count 0 Nov 29 23:25:48.573: INFO: pod1 started at 2022-11-29 23:25:38 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.573: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:25:48.573: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:25:40 +0000 UTC (0+7 container statuses recorded) Nov 29 23:25:48.573: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:48.573: INFO: webserver-57d64b6f67-8zn5s started at 2022-11-29 23:25:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.573: INFO: Container httpd ready: true, restart count 0 Nov 29 23:25:48.573: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:48.573: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:48.573: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:25:48.573: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:25:48.573: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:25:48.573: INFO: pause-pod-1 started at 2022-11-29 23:25:25 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.573: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:25:48.573: INFO: pod-ebd5b434-5ace-45dd-b471-bc184b05fffb started at 2022-11-29 23:25:44 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:48.573: INFO: Container write-pod ready: false, restart count 0 Nov 29 23:25:48.980: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:25:48.980: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:25:49.010: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 15308 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1658":"csi-mock-csi-mock-volumes-1658","ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-29 23:24:52 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:24:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:25:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:25:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1658^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1658^4,DevicePath:,},},Config:nil,},} Nov 29 23:25:49.010: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:25:49.045: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:25:49.084: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:49.084: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:25:49.084: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:49.084: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:49.084: INFO: pvc-volume-tester-69z92 started at 2022-11-29 23:24:51 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container volume-tester ready: true, restart count 0 Nov 29 23:25:49.084: INFO: hostexec-ip-172-20-43-249.us-east-2.compute.internal-n7tcn started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:25:49.084: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:25:49.084: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:25:49.084: INFO: test-deployment-6c9dc9cdf6-ppp52 started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container test-deployment ready: true, restart count 0 Nov 29 23:25:49.084: INFO: deployment-585449566-ftgcv started at 2022-11-29 23:24:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container nginx ready: false, restart count 0 Nov 29 23:25:49.084: INFO: liveness-71a84166-20f1-434f-bf37-14c767653a80 started at 2022-11-29 23:24:23 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container agnhost-container ready: true, restart count 4 Nov 29 23:25:49.084: INFO: csi-mockplugin-attacher-0 started at 2022-11-29 23:24:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:25:49.084: INFO: ss2-0 started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container webserver ready: true, restart count 0 Nov 29 23:25:49.084: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container coredns ready: true, restart count 0 Nov 29 23:25:49.084: INFO: webserver-78f4dcd76f-vrf5m started at 2022-11-29 23:25:31 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container httpd ready: true, restart count 0 Nov 29 23:25:49.084: INFO: csi-mockplugin-0 started at 2022-11-29 23:24:35 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:49.084: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:25:49.084: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:25:49.084: INFO: Container mock ready: true, restart count 0 Nov 29 23:25:49.084: INFO: webserver-57d64b6f67-cdp76 started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container httpd ready: true, restart count 0 Nov 29 23:25:49.084: INFO: test-deployment-56c98d85f9-jc54b started at 2022-11-29 23:25:46 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.084: INFO: Container test-deployment ready: false, restart count 0 Nov 29 23:25:49.307: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:25:49.307: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:25:49.338: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 14217 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:24:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:24:29 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:24:29 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:24:29 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:24:29 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:25:49.339: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:25:49.370: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:25:49.418: INFO: webserver-57d64b6f67-477gm started at <nil> (0+0 container statuses recorded) Nov 29 23:25:49.418: INFO: pod-adoption-release started at 2022-11-29 23:25:46 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container pod-adoption-release ready: false, restart count 0 Nov 29 23:25:49.418: INFO: deployment-585449566-t6rmc started at 2022-11-29 23:24:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container nginx ready: false, restart count 0 Nov 29 23:25:49.418: INFO: pod-prestop-hook-9909c0d5-b39a-4209-b89b-5bbafc5f96b3 started at 2022-11-29 23:25:11 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container nginx ready: true, restart count 0 Nov 29 23:25:49.418: INFO: hostexec-ip-172-20-58-90.us-east-2.compute.internal-hpmnr started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:25:49.418: INFO: deployment-585449566-kmpmx started at 2022-11-29 23:24:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container nginx ready: false, restart count 0 Nov 29 23:25:49.418: INFO: deployment-55649fd747-nrrgq started at 2022-11-29 23:24:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container nginx ready: false, restart count 0 Nov 29 23:25:49.418: INFO: test-deployment-6c9dc9cdf6-cfbdh started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container test-deployment ready: true, restart count 0 Nov 29 23:25:49.418: INFO: inline-volume-txwvq started at 2022-11-29 23:25:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container volume-tester ready: false, restart count 0 Nov 29 23:25:49.418: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:25:49.418: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:25:49.418: INFO: webserver-57d64b6f67-2hzlw started at 2022-11-29 23:25:43 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container httpd ready: true, restart count 0 Nov 29 23:25:49.418: INFO: ss2-1 started at 2022-11-29 23:25:47 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container webserver ready: false, restart count 0 Nov 29 23:25:49.418: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:25:49.418: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:25:49.418: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:25:49.418: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:25:49.418: INFO: execpodd27sd started at 2022-11-29 23:25:42 +0000 UTC (0+1 container statuses recorded) Nov 29 23:25:49.418: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:25:49.861: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:25:49.861: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-1069" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 29 23:38:52.898: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003fe50f0>: { s: "pod \"pod-4bdadfa2-ea1c-4e98-9285-94227d41b913\" is not Running: timed out waiting for the condition", } pod "pod-4bdadfa2-ea1c-4e98-9285-94227d41b913" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_06.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:33:52.536: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 29 23:33:52.683: INFO: Creating resource for dynamic PV Nov 29 23:33:52.683: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-6290-e2e-scwsdhl �[1mSTEP�[0m: creating a claim Nov 29 23:33:52.713: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 29 23:38:52.897: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003fe50f0>: { s: "pod \"pod-4bdadfa2-ea1c-4e98-9285-94227d41b913\" is not Running: timed out waiting for the condition", } pod "pod-4bdadfa2-ea1c-4e98-9285-94227d41b913" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000227680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000227680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000227680, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 29 23:38:52.898: INFO: Deleting pod "pod-4bdadfa2-ea1c-4e98-9285-94227d41b913" in namespace "volume-expand-6290" Nov 29 23:38:52.933: INFO: Wait up to 5m0s for pod "pod-4bdadfa2-ea1c-4e98-9285-94227d41b913" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 29 23:38:52.962: INFO: Deleting pod "pod-4bdadfa2-ea1c-4e98-9285-94227d41b913" in namespace "volume-expand-6290" �[1mSTEP�[0m: Deleting pvc Nov 29 23:38:53.019: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comg5lp6" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-6290". �[1mSTEP�[0m: Found 4 events. Nov 29 23:38:53.112: INFO: At 2022-11-29 23:33:52 +0000 UTC - event for ebs.csi.aws.comg5lp6: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:38:53.112: INFO: At 2022-11-29 23:33:52 +0000 UTC - event for ebs.csi.aws.comg5lp6: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:38:53.112: INFO: At 2022-11-29 23:33:52 +0000 UTC - event for ebs.csi.aws.comg5lp6: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-6290/ebs.csi.aws.comg5lp6" Nov 29 23:38:53.112: INFO: At 2022-11-29 23:34:02 +0000 UTC - event for ebs.csi.aws.comg5lp6: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-6290-e2e-scwsdhl": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:38:53.141: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:38:53.141: INFO: Nov 29 23:38:53.170: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:38:53.202: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 42024 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-29 23:38:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:38:53.203: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:38:53.234: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:38:53.268: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:38:53.268: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:38:53.268: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:53.268: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:53.268: INFO: netserver-0 started at 2022-11-29 23:38:36 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Container webserver ready: false, restart count 0 Nov 29 23:38:53.268: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:38:53.268: INFO: pod-configmaps-c7bcea27-07b7-4f50-822d-4215de22bab0 started at 2022-11-29 23:37:29 +0000 UTC (0+3 container statuses recorded) Nov 29 23:38:53.268: INFO: Container createcm-volume-test ready: true, restart count 0 Nov 29 23:38:53.268: INFO: Container delcm-volume-test ready: true, restart count 0 Nov 29 23:38:53.268: INFO: Container updcm-volume-test ready: true, restart count 0 Nov 29 23:38:53.268: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Container coredns ready: true, restart count 0 Nov 29 23:38:53.268: INFO: hostexec-ip-172-20-34-50.us-east-2.compute.internal-6tj77 started at 2022-11-29 23:38:25 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:38:53.268: INFO: pod-subpath-test-preprovisionedpv-hlrn started at 2022-11-29 23:38:31 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Init container init-volume-preprovisionedpv-hlrn ready: true, restart count 0 Nov 29 23:38:53.268: INFO: Container test-container-subpath-preprovisionedpv-hlrn ready: true, restart count 0 Nov 29 23:38:53.268: INFO: pod-terminate-status-0-5 started at 2022-11-29 23:38:49 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Init container fail ready: false, restart count 0 Nov 29 23:38:53.268: INFO: Container blocked ready: false, restart count 0 Nov 29 23:38:53.268: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:38:53.268: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:38:53.268: INFO: agnhost-primary-2zj6r started at 2022-11-29 23:37:12 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Container agnhost-primary ready: false, restart count 0 Nov 29 23:38:53.268: INFO: pod-qos-class-39fe6c30-3ffe-4b33-bccf-d6a4e3cb7e51 started at 2022-11-29 23:37:18 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:53.268: INFO: Container agnhost ready: false, restart count 0 Nov 29 23:38:54.445: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:38:54.445: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:38:54.475: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 40183 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:38:54.476: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:38:54.508: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:38:54.545: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:38:54.545: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:38:54.545: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:38:54.545: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:54.545: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:38:54.545: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:38:54.545: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:38:54.545: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:54.545: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:38:54.545: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:38:54.545: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:38:54.545: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:38:54.545: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:38:54.545: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:38:54.545: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.545: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:38:54.727: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:38:54.727: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:38:54.756: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 40637 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-930":"ip-172-20-40-36.us-east-2.compute.internal","ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-29 23:36:35 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:36:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:48 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:48 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:48 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:48 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-930^ce75998a-703e-11ed-8ca6-b6e8b75ecdb0],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-930^ce75998a-703e-11ed-8ca6-b6e8b75ecdb0,DevicePath:,},},Config:nil,},} Nov 29 23:38:54.757: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:38:54.789: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:38:54.851: INFO: netserver-1 started at 2022-11-29 23:38:36 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.852: INFO: Container webserver ready: false, restart count 0 Nov 29 23:38:54.852: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:38:54.852: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:54.852: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:54.852: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:38:54.852: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:37:32 +0000 UTC (0+7 container statuses recorded) Nov 29 23:38:54.852: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:54.852: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:54.852: INFO: pod-ee15b4eb-c5d8-49ef-abcd-91d2d9fad89b started at 2022-11-29 23:37:39 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.852: INFO: Container write-pod ready: true, restart count 0 Nov 29 23:38:54.852: INFO: busybox-c3945811-0a68-45e3-9fd9-3b165f5496fd started at 2022-11-29 23:36:21 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:54.852: INFO: Container busybox ready: true, restart count 0 Nov 29 23:38:54.852: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:38:54 +0000 UTC (0+7 container statuses recorded) Nov 29 23:38:54.852: INFO: Container csi-attacher ready: false, restart count 0 Nov 29 23:38:54.852: INFO: Container csi-provisioner ready: false, restart count 0 Nov 29 23:38:54.852: INFO: Container csi-resizer ready: false, restart count 0 Nov 29 23:38:54.852: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 29 23:38:54.852: INFO: Container hostpath ready: false, restart count 0 Nov 29 23:38:54.852: INFO: Container liveness-probe ready: false, restart count 0 Nov 29 23:38:54.852: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 29 23:38:55.098: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:38:55.098: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:38:55.129: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 41971 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5971":"ip-172-20-43-249.us-east-2.compute.internal","ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-29 23:38:30 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:38:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:38:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-5971^eb2a5cdf-703e-11ed-a7f8-ce5c09ca4903],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-5971^eb2a5cdf-703e-11ed-a7f8-ce5c09ca4903,DevicePath:,},},Config:nil,},} Nov 29 23:38:55.129: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:38:55.160: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:38:55.200: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:38:55.201: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:55.201: INFO: inline-volume-tester-69lbb started at 2022-11-29 23:38:29 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 29 23:38:55.201: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:38:55.201: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:38:21 +0000 UTC (0+7 container statuses recorded) Nov 29 23:38:55.201: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:55.201: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:55.201: INFO: suspend-true-to-false-kd7k5 started at 2022-11-29 23:38:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Container c ready: false, restart count 0 Nov 29 23:38:55.201: INFO: concurrent-27829418-xp292 started at 2022-11-29 23:38:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Container c ready: true, restart count 0 Nov 29 23:38:55.201: INFO: netserver-2 started at 2022-11-29 23:38:36 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Container webserver ready: false, restart count 0 Nov 29 23:38:55.201: INFO: pod-terminate-status-2-6 started at 2022-11-29 23:38:54 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Init container fail ready: false, restart count 0 Nov 29 23:38:55.201: INFO: Container blocked ready: false, restart count 0 Nov 29 23:38:55.201: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.201: INFO: Container coredns ready: true, restart count 0 Nov 29 23:38:55.568: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:38:55.568: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:38:55.597: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 38660 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:36:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:38:55.597: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:38:55.629: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:38:55.664: INFO: hostexec-ip-172-20-58-90.us-east-2.compute.internal-qpp7b started at 2022-11-29 23:38:29 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:38:55.665: INFO: netserver-3 started at 2022-11-29 23:38:36 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Container webserver ready: false, restart count 0 Nov 29 23:38:55.665: INFO: pod-terminate-status-1-5 started at 2022-11-29 23:38:50 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Init container fail ready: false, restart count 0 Nov 29 23:38:55.665: INFO: Container blocked ready: false, restart count 0 Nov 29 23:38:55.665: INFO: downwardapi-volume-f0f93324-b88e-47e5-b177-6f5f2d2cc035 started at 2022-11-29 23:38:51 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Container client-container ready: false, restart count 0 Nov 29 23:38:55.665: INFO: hostexec-ip-172-20-58-90.us-east-2.compute.internal-5rwbj started at 2022-11-29 23:38:44 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:38:55.665: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:38:55.665: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:38:55.665: INFO: busybox-5b3ae421-034f-4e1b-b0e0-8b128938f0a8 started at 2022-11-29 23:38:26 +0000 UTC (0+1 container statuses recorded) Nov 29 23:38:55.665: INFO: Container busybox ready: true, restart count 0 Nov 29 23:38:55.665: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:38:55.665: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:38:55.665: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:38:55.665: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:38:55.665: INFO: suspend-true-to-false-qkw8v started at <nil> (0+0 container statuses recorded) Nov 29 23:38:58.499: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:38:58.499: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-6290" for this suite.
Find pod-4bdadfa2-ea1c-4e98-9285-94227d41b913 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 29 23:28:26.163: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003bdd1f0>: { s: "pod \"pod-28645711-e001-4fed-b1d3-47928cf77c3a\" is not Running: timed out waiting for the condition", } pod "pod-28645711-e001-4fed-b1d3-47928cf77c3a" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_22.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:23:25.790: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 29 23:23:25.938: INFO: Creating resource for dynamic PV Nov 29 23:23:25.938: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-1433-e2e-scc2rrj �[1mSTEP�[0m: creating a claim Nov 29 23:23:25.969: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 29 23:28:26.163: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003bdd1f0>: { s: "pod \"pod-28645711-e001-4fed-b1d3-47928cf77c3a\" is not Running: timed out waiting for the condition", } pod "pod-28645711-e001-4fed-b1d3-47928cf77c3a" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000c86480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000c86480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000c86480, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 29 23:28:26.163: INFO: Deleting pod "pod-28645711-e001-4fed-b1d3-47928cf77c3a" in namespace "volume-expand-1433" Nov 29 23:28:26.196: INFO: Wait up to 5m0s for pod "pod-28645711-e001-4fed-b1d3-47928cf77c3a" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 29 23:28:26.227: INFO: Deleting pod "pod-28645711-e001-4fed-b1d3-47928cf77c3a" in namespace "volume-expand-1433" �[1mSTEP�[0m: Deleting pvc Nov 29 23:28:26.285: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comfpfd5" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-1433". �[1mSTEP�[0m: Found 5 events. Nov 29 23:28:26.384: INFO: At 2022-11-29 23:23:25 +0000 UTC - event for ebs.csi.aws.comfpfd5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:28:26.384: INFO: At 2022-11-29 23:23:26 +0000 UTC - event for ebs.csi.aws.comfpfd5: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:28:26.384: INFO: At 2022-11-29 23:23:26 +0000 UTC - event for ebs.csi.aws.comfpfd5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-1433/ebs.csi.aws.comfpfd5" Nov 29 23:28:26.384: INFO: At 2022-11-29 23:23:36 +0000 UTC - event for ebs.csi.aws.comfpfd5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-1433-e2e-scc2rrj": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:28:26.384: INFO: At 2022-11-29 23:23:47 +0000 UTC - event for ebs.csi.aws.comfpfd5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-1433-e2e-scc2rrj": rpc error: code = Internal desc = Could not create volume "pvc-49095a89-490e-4f20-8ae2-918407b487c1": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 29 23:28:26.416: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:28:26.416: INFO: Nov 29 23:28:26.452: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:28:26.486: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 21375 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-29 23:27:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:28:26.487: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:28:26.523: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:28:26.567: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.567: INFO: Container coredns ready: true, restart count 0 Nov 29 23:28:26.567: INFO: hostexec-ip-172-20-34-50.us-east-2.compute.internal-ttwzm started at 2022-11-29 23:28:15 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.567: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:28:26.567: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:28:26.567: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:28:26.567: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:28:26.567: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:28:26.567: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:28:26.567: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:28:26.567: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:28:26.567: INFO: test-rolling-update-with-lb-7db9cf5ff8-ct5b6 started at 2022-11-29 23:26:44 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.567: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:28:26.567: INFO: webserver-pod started at 2022-11-29 23:25:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.567: INFO: Container agnhost ready: false, restart count 0 Nov 29 23:28:26.567: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.567: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:28:26.567: INFO: csi-mockplugin-0 started at 2022-11-29 23:27:39 +0000 UTC (0+4 container statuses recorded) Nov 29 23:28:26.567: INFO: Container busybox ready: true, restart count 0 Nov 29 23:28:26.567: INFO: Container csi-provisioner ready: false, restart count 1 Nov 29 23:28:26.567: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:28:26.567: INFO: Container mock ready: true, restart count 0 Nov 29 23:28:26.748: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:28:26.748: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:28:26.778: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 20634 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:28:26.778: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:28:26.812: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:28:26.881: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:28:26.881: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:28:26.881: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:28:26.881: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:28:26.881: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:28:26.881: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:28:26.881: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:28:26.881: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:28:26.881: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:28:26.881: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:28:26.881: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:28:26.881: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:26.881: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:28:26.881: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:28:26.881: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:28:26.881: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:28:27.116: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:28:27.116: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:28:27.146: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 21621 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1590":"csi-mock-csi-mock-volumes-1590","ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-29 23:28:07 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:28:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:28:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1590^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1590^4,DevicePath:,},},Config:nil,},} Nov 29 23:28:27.147: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:28:27.183: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:28:27.217: INFO: test-rolling-update-with-lb-7db9cf5ff8-7h5qk started at 2022-11-29 23:26:44 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.217: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:28:27.217: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:28:27.217: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:28:27.217: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:28:27.217: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:28:27.217: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:28:27.217: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:28:27.217: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:28:27.217: INFO: adopt-release-bmc4z started at 2022-11-29 23:27:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.217: INFO: Container c ready: false, restart count 0 Nov 29 23:28:27.217: INFO: csi-mockplugin-0 started at 2022-11-29 23:28:01 +0000 UTC (0+3 container statuses recorded) Nov 29 23:28:27.217: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:28:27.217: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:28:27.217: INFO: Container mock ready: true, restart count 0 Nov 29 23:28:27.217: INFO: csi-mockplugin-attacher-0 started at 2022-11-29 23:28:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.217: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:28:27.217: INFO: implicit-nonroot-uid started at <nil> (0+0 container statuses recorded) Nov 29 23:28:27.399: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:28:27.399: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:28:27.446: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 21221 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:27:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:28:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:28:27.447: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:28:27.506: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:28:27.570: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:28:27.570: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:28:27.570: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:28:27.570: INFO: successful-jobs-history-limit-27829408-9vj4c started at 2022-11-29 23:28:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.570: INFO: Container c ready: false, restart count 0 Nov 29 23:28:27.570: INFO: probe-test-55ef8fa9-9747-467f-9251-7e679d15c886 started at 2022-11-29 23:28:12 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.570: INFO: Container probe-test-55ef8fa9-9747-467f-9251-7e679d15c886 ready: false, restart count 0 Nov 29 23:28:27.570: INFO: pod-projected-configmaps-bdaa3f9a-eca3-44be-926d-0022d88626f1 started at 2022-11-29 23:28:23 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.570: INFO: Container agnhost-container ready: false, restart count 0 Nov 29 23:28:27.570: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.570: INFO: Container coredns ready: true, restart count 0 Nov 29 23:28:27.570: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:28:27.570: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:28:27.570: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:28:27.570: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:28:27.570: INFO: hostexec-ip-172-20-43-249.us-east-2.compute.internal-f6gkz started at 2022-11-29 23:28:11 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.570: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:28:27.727: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:28:27.727: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:28:27.757: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 20534 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:24:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:28:27.757: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:28:27.791: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:28:27.824: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:28:27.824: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:28:27.824: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:28:27.824: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:28:27.824: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:28:27.824: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:28:27.824: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:28:27.824: INFO: test-webserver-8f294c7e-7a26-4a12-a2d9-28c256c07264 started at 2022-11-29 23:28:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.824: INFO: Container test-webserver ready: true, restart count 0 Nov 29 23:28:27.824: INFO: pod-logs-websocket-8349f7f0-730c-4dd0-a0e4-a0ce3f8dd472 started at 2022-11-29 23:28:08 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.824: INFO: Container main ready: true, restart count 0 Nov 29 23:28:27.824: INFO: test-rolling-update-with-lb-7db9cf5ff8-xb7qb started at 2022-11-29 23:26:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:28:27.824: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:28:27.970: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:28:27.970: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-1433" for this suite.
Find pod-28645711-e001-4fed-b1d3-47928cf77c3a mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:45:29.574: Unexpected error: <*errors.errorString | 0xc00106e6e0>: { s: "pod \"pod-48aeda91-8416-4b53-9041-770a465b66ab\" is not Running: timed out waiting for the condition", } pod "pod-48aeda91-8416-4b53-9041-770a465b66ab" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_10.xml
{"msg":"PASSED [sig-cli] Kubectl client Kubectl get componentstatuses should get componentstatuses","total":-1,"completed":38,"skipped":349,"failed":1,"failures":["External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand Verify if offline PVC expansion works"]} [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:40:29.210: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:40:29.356: INFO: Creating resource for dynamic PV Nov 29 23:40:29.356: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-3209-e2e-sck2pws �[1mSTEP�[0m: creating a claim Nov 29 23:40:29.386: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-3209 with fsgroup 1000 Nov 29 23:45:29.574: FAIL: Unexpected error: <*errors.errorString | 0xc00106e6e0>: { s: "pod \"pod-48aeda91-8416-4b53-9041-770a465b66ab\" is not Running: timed out waiting for the condition", } pod "pod-48aeda91-8416-4b53-9041-770a465b66ab" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001567760, 0xc001d170d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000227b00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000227b00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000227b00, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 29 23:45:29.604: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com7rk7n" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-3209". �[1mSTEP�[0m: Found 4 events. Nov 29 23:45:29.695: INFO: At 2022-11-29 23:40:29 +0000 UTC - event for ebs.csi.aws.com7rk7n: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:45:29.695: INFO: At 2022-11-29 23:40:29 +0000 UTC - event for ebs.csi.aws.com7rk7n: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:45:29.695: INFO: At 2022-11-29 23:40:29 +0000 UTC - event for ebs.csi.aws.com7rk7n: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-3209/ebs.csi.aws.com7rk7n" Nov 29 23:45:29.695: INFO: At 2022-11-29 23:40:39 +0000 UTC - event for ebs.csi.aws.com7rk7n: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-3209-e2e-sck2pws": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:45:29.724: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:45:29.724: INFO: pod-48aeda91-8416-4b53-9041-770a465b66ab Pending [] Nov 29 23:45:29.724: INFO: Nov 29 23:45:29.755: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:29.784: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 46970 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:38:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:43:27 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:29.784: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:29.816: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:29.849: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:29.849: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:29.849: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:29.849: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:29.849: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:29.849: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:45:29.849: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:29.849: INFO: Container coredns ready: true, restart count 0 Nov 29 23:45:29.849: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:29.849: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:29.849: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:30.039: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:45:30.040: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:30.072: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 46590 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:42:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:30.072: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:30.108: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:30.150: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:45:30.150: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:45:30.150: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:30.150: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:30.150: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:45:30.150: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:45:30.150: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:45:30.150: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:45:30.150: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:45:30.150: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:45:30.150: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:30.150: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.150: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:45:30.150: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:45:30.150: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:30.150: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:30.409: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:45:30.409: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:30.447: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 47319 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:44:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:30.447: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:30.480: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:30.513: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:30.513: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:30.513: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:30.513: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:30.513: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:30.513: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:30.513: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:30.687: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:45:30.687: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:30.717: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 47237 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:43:57 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:30.717: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:30.749: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:30.781: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:45:30.781: INFO: Container coredns ready: true, restart count 0 Nov 29 23:45:30.781: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:30.781: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:30.781: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:30.781: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:30.781: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:30.781: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:30.781: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:30.947: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:45:30.947: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:30.976: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 46968 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:36:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:42:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:45:30.977: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:31.008: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:31.040: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:45:31.040: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:45:31.040: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:45:31.040: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:45:31.040: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:45:31.040: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:45:31.040: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:45:31.190: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:45:31.190: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-3209" for this suite.
Find pod-48aeda91-8416-4b53-9041-770a465b66ab mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:31:51.347: Unexpected error: <*errors.errorString | 0xc004b149a0>: { s: "pod \"pod-9155b474-f072-42ce-ba23-e4766208b089\" is not Running: timed out waiting for the condition", } pod "pod-9155b474-f072-42ce-ba23-e4766208b089" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_24.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:26:50.983: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:26:51.132: INFO: Creating resource for dynamic PV Nov 29 23:26:51.132: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-5534-e2e-scx4fzl �[1mSTEP�[0m: creating a claim Nov 29 23:26:51.162: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-5534 with fsgroup 1000 Nov 29 23:31:51.347: FAIL: Unexpected error: <*errors.errorString | 0xc004b149a0>: { s: "pod \"pod-9155b474-f072-42ce-ba23-e4766208b089\" is not Running: timed out waiting for the condition", } pod "pod-9155b474-f072-42ce-ba23-e4766208b089" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001d95e40, 0xc0036cb0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0004d6600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0004d6600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0004d6600, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 29 23:31:51.377: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com8xt8m" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-5534". �[1mSTEP�[0m: Found 4 events. Nov 29 23:31:51.471: INFO: At 2022-11-29 23:26:51 +0000 UTC - event for ebs.csi.aws.com8xt8m: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:31:51.471: INFO: At 2022-11-29 23:26:51 +0000 UTC - event for ebs.csi.aws.com8xt8m: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:31:51.471: INFO: At 2022-11-29 23:26:51 +0000 UTC - event for ebs.csi.aws.com8xt8m: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-5534/ebs.csi.aws.com8xt8m" Nov 29 23:31:51.471: INFO: At 2022-11-29 23:27:01 +0000 UTC - event for ebs.csi.aws.com8xt8m: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5534-e2e-scx4fzl": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:31:51.500: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:31:51.500: INFO: pod-9155b474-f072-42ce-ba23-e4766208b089 Pending [] Nov 29 23:31:51.500: INFO: Nov 29 23:31:51.531: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:51.561: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 26881 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4453":"csi-mock-csi-mock-volumes-4453","ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-29 23:29:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:51.562: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:51.594: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:51.639: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.639: INFO: Container coredns ready: true, restart count 0 Nov 29 23:31:51.639: INFO: startup-script started at 2022-11-29 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.639: INFO: Container startup-script ready: true, restart count 0 Nov 29 23:31:51.639: INFO: pvc-volume-tester-99dcf started at 2022-11-29 23:31:49 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.639: INFO: Container volume-tester ready: true, restart count 0 Nov 29 23:31:51.639: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:51.639: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:51.639: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:51.639: INFO: send-events-5b31c90b-6a20-4b6f-8d93-020c10dd0131 started at 2022-11-29 23:31:16 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.639: INFO: Container p ready: true, restart count 0 Nov 29 23:31:51.639: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:51.639: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:51.639: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:51.639: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:51.639: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.639: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:31:51.639: INFO: csi-mockplugin-0 started at 2022-11-29 23:31:44 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:51.639: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:31:51.639: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:31:51.639: INFO: Container mock ready: true, restart count 0 Nov 29 23:31:51.811: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:31:51.812: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:51.841: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 20634 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:51.842: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:51.881: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:51.945: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:31:51.945: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:51.945: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:51.945: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:31:51.945: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:31:51.945: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:31:51.945: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:31:51.945: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:31:51.945: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:51.945: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:51.945: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:31:51.945: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:31:51.945: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:51.945: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:51.945: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:31:51.945: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:31:51.946: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:31:52.224: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:31:52.224: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:52.270: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 26823 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:28:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:52.270: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:52.307: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:52.339: INFO: pod-ephm-test-projected-vxrn started at 2022-11-29 23:30:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:52.339: INFO: Container test-container-subpath-projected-vxrn ready: false, restart count 0 Nov 29 23:31:52.339: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:52.339: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:52.339: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:52.339: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:52.339: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:52.339: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:52.339: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:52.339: INFO: pod-subpath-test-configmap-dn2z started at 2022-11-29 23:31:49 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:52.339: INFO: Container test-container-subpath-configmap-dn2z ready: true, restart count 0 Nov 29 23:31:52.489: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:31:52.489: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:52.519: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 24907 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:30:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:52.520: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:52.551: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:52.585: INFO: rs-9bpbh started at 2022-11-29 23:30:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:52.585: INFO: Container donothing ready: false, restart count 0 Nov 29 23:31:52.585: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:52.585: INFO: Container coredns ready: true, restart count 0 Nov 29 23:31:52.585: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:52.585: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:52.585: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:52.585: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:52.585: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:52.585: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:52.585: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:52.585: INFO: hostexec-ip-172-20-43-249.us-east-2.compute.internal-2dhzd started at 2022-11-29 23:31:38 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:52.585: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:31:52.585: INFO: pod-secrets-458971a2-cfa1-4a29-a534-a834593918ec started at 2022-11-29 23:30:25 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:52.585: INFO: Container creates-volume-test ready: true, restart count 0 Nov 29 23:31:52.585: INFO: Container dels-volume-test ready: true, restart count 0 Nov 29 23:31:52.585: INFO: Container upds-volume-test ready: true, restart count 0 Nov 29 23:31:53.687: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:31:53.687: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:53.716: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 26670 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:24:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:31:09 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:31:53.717: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:53.748: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:53.783: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:31:52 +0000 UTC (0+7 container statuses recorded) Nov 29 23:31:53.783: INFO: Container csi-attacher ready: false, restart count 0 Nov 29 23:31:53.783: INFO: Container csi-provisioner ready: false, restart count 0 Nov 29 23:31:53.783: INFO: Container csi-resizer ready: false, restart count 0 Nov 29 23:31:53.783: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 29 23:31:53.783: INFO: Container hostpath ready: false, restart count 0 Nov 29 23:31:53.783: INFO: Container liveness-probe ready: false, restart count 0 Nov 29 23:31:53.783: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 29 23:31:53.783: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:31:53.783: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:31:53.783: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:31:53.783: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:31:53.783: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:31:53.783: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:31:53.783: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:31:53.783: INFO: boom-server started at 2022-11-29 23:30:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:53.783: INFO: Container boom-server ready: true, restart count 0 Nov 29 23:31:53.783: INFO: test-webserver-8f294c7e-7a26-4a12-a2d9-28c256c07264 started at 2022-11-29 23:28:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:31:53.783: INFO: Container test-webserver ready: true, restart count 0 Nov 29 23:31:54.037: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:31:54.037: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-5534" for this suite.
Find pod-9155b474-f072-42ce-ba23-e4766208b089 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:27:27.769: Unexpected error: <*errors.errorString | 0xc0022fda60>: { s: "pod \"pod-b3dba713-5cc8-4674-9829-d6aa03a1d037\" is not Running: timed out waiting for the condition", } pod "pod-b3dba713-5cc8-4674-9829-d6aa03a1d037" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_06.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:22:27.394: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:22:27.551: INFO: Creating resource for dynamic PV Nov 29 23:22:27.551: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-8715-e2e-sc6p82d �[1mSTEP�[0m: creating a claim Nov 29 23:22:27.583: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-8715 with fsgroup 1000 Nov 29 23:27:27.769: FAIL: Unexpected error: <*errors.errorString | 0xc0022fda60>: { s: "pod \"pod-b3dba713-5cc8-4674-9829-d6aa03a1d037\" is not Running: timed out waiting for the condition", } pod "pod-b3dba713-5cc8-4674-9829-d6aa03a1d037" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0042eb4a0, 0xc0039690d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000227680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000227680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000227680, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 29 23:27:27.799: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comp65kr" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-8715". �[1mSTEP�[0m: Found 4 events. Nov 29 23:27:27.888: INFO: At 2022-11-29 23:22:27 +0000 UTC - event for ebs.csi.aws.comp65kr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:27:27.888: INFO: At 2022-11-29 23:22:27 +0000 UTC - event for ebs.csi.aws.comp65kr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:27:27.888: INFO: At 2022-11-29 23:22:27 +0000 UTC - event for ebs.csi.aws.comp65kr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-8715/ebs.csi.aws.comp65kr" Nov 29 23:27:27.888: INFO: At 2022-11-29 23:22:37 +0000 UTC - event for ebs.csi.aws.comp65kr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8715-e2e-sc6p82d": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:27:27.917: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:27:27.918: INFO: pod-b3dba713-5cc8-4674-9829-d6aa03a1d037 Pending [] Nov 29 23:27:27.918: INFO: Nov 29 23:27:27.948: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:27:27.977: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 18976 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:26:36 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:27:27.978: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:27:28.009: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:27:28.048: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:27:28.048: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:27:28.048: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:27:28.048: INFO: busybox-aed6d519-0fb9-483b-9e8d-ee90987e3948 started at 2022-11-29 23:27:06 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.048: INFO: Container busybox ready: true, restart count 0 Nov 29 23:27:28.048: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:27:28.048: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:27:28.048: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:28.048: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:27:28.048: INFO: test-rolling-update-with-lb-7db9cf5ff8-ct5b6 started at 2022-11-29 23:26:44 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.048: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:27:28.049: INFO: webserver-pod started at 2022-11-29 23:25:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.049: INFO: Container agnhost ready: false, restart count 0 Nov 29 23:27:28.049: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.049: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:27:28.049: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.049: INFO: Container coredns ready: true, restart count 0 Nov 29 23:27:28.049: INFO: deployment-55649fd747-jzrz6 started at 2022-11-29 23:25:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.049: INFO: Container nginx ready: false, restart count 0 Nov 29 23:27:28.189: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:27:28.190: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:27:28.219: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 8195 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:22:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:27:28.219: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:27:28.251: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:27:28.325: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:27:28.325: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:27:28.325: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:27:28.325: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:27:28.325: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:27:28.325: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:27:28.325: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:27:28.325: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:27:28.325: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:27:28.325: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:28.325: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:27:28.325: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:27:28.325: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:27:28.325: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:27:28.325: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.325: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:27:28.480: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:27:28.480: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:27:28.509: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 19365 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-8949":"ip-172-20-40-36.us-east-2.compute.internal","ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:26:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:26:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:27:28.510: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:27:28.542: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:27:28.575: INFO: deployment-55649fd747-b988q started at 2022-11-29 23:25:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.575: INFO: Container nginx ready: false, restart count 0 Nov 29 23:27:28.575: INFO: startup-9820f8d0-66c8-4eb0-8920-c9ba80cc6cb8 started at 2022-11-29 23:26:29 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.575: INFO: Container busybox ready: true, restart count 0 Nov 29 23:27:28.575: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:26:45 +0000 UTC (0+7 container statuses recorded) Nov 29 23:27:28.575: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:27:28.575: INFO: test-rolling-update-with-lb-7db9cf5ff8-7h5qk started at 2022-11-29 23:26:44 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.575: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:27:28.575: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:27:28.575: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:27:28.575: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:27:28.575: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:27:28.575: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:27:28.749: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:27:28.749: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:27:28.778: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 20281 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:27:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:16 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:16 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:16 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:16 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:27:28.779: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:27:28.810: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:27:28.843: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:27:28.844: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:27:28.844: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:28.844: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:27:28.844: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:27:28.844: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:27:28.844: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:27:28.844: INFO: deployment-55649fd747-kk2g4 started at 2022-11-29 23:25:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.844: INFO: Container nginx ready: false, restart count 0 Nov 29 23:27:28.844: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:28.844: INFO: Container coredns ready: true, restart count 0 Nov 29 23:27:29.024: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:27:29.024: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:27:29.054: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 19085 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:24:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:26:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:26:39 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:27:29.054: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:27:29.086: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:27:29.118: INFO: test-rolling-update-with-lb-7db9cf5ff8-xb7qb started at 2022-11-29 23:26:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:29.118: INFO: Container agnhost ready: true, restart count 0 Nov 29 23:27:29.118: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:27:29.118: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:27:29.118: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:27:29.118: INFO: deployment-55649fd747-v45jz started at 2022-11-29 23:25:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:29.118: INFO: Container nginx ready: false, restart count 0 Nov 29 23:27:29.118: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:27:29.118: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:27:29.118: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:27:29.118: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:27:29.118: INFO: liveness-d255549e-fc99-43f6-8b9e-3cdf05eb450f started at 2022-11-29 23:27:24 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:29.118: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:27:29.118: INFO: deployment-55649fd747-wpk6p started at 2022-11-29 23:25:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:27:29.118: INFO: Container nginx ready: false, restart count 0 Nov 29 23:27:29.256: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:27:29.256: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-8715" for this suite.
Find pod-b3dba713-5cc8-4674-9829-d6aa03a1d037 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:41:09.071: Unexpected error: <*errors.errorString | 0xc00468ad00>: { s: "pod \"pod-f487d8c4-39ed-4b0e-a4ae-1e2d7c8b7a8a\" is not Running: timed out waiting for the condition", } pod "pod-f487d8c4-39ed-4b0e-a4ae-1e2d7c8b7a8a" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_21.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:36:08.699: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:36:08.851: INFO: Creating resource for dynamic PV Nov 29 23:36:08.851: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-9554-e2e-scblpkd �[1mSTEP�[0m: creating a claim Nov 29 23:36:08.882: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-9554 with fsgroup 1000 Nov 29 23:41:09.071: FAIL: Unexpected error: <*errors.errorString | 0xc00468ad00>: { s: "pod \"pod-f487d8c4-39ed-4b0e-a4ae-1e2d7c8b7a8a\" is not Running: timed out waiting for the condition", } pod "pod-f487d8c4-39ed-4b0e-a4ae-1e2d7c8b7a8a" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0044c18c0, 0xc00515f0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000c5f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000c5f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000c5f980, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 29 23:41:09.101: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comt6nbc" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-9554". �[1mSTEP�[0m: Found 4 events. Nov 29 23:41:09.192: INFO: At 2022-11-29 23:36:08 +0000 UTC - event for ebs.csi.aws.comt6nbc: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:41:09.192: INFO: At 2022-11-29 23:36:08 +0000 UTC - event for ebs.csi.aws.comt6nbc: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:41:09.192: INFO: At 2022-11-29 23:36:08 +0000 UTC - event for ebs.csi.aws.comt6nbc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-9554/ebs.csi.aws.comt6nbc" Nov 29 23:41:09.192: INFO: At 2022-11-29 23:36:18 +0000 UTC - event for ebs.csi.aws.comt6nbc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-9554-e2e-scblpkd": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:41:09.222: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:41:09.222: INFO: pod-f487d8c4-39ed-4b0e-a4ae-1e2d7c8b7a8a Pending [] Nov 29 23:41:09.222: INFO: Nov 29 23:41:09.253: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.282: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 43722 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:38:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{},"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:09.283: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.315: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.349: INFO: liveness-a080bfd2-2aa6-4263-885d-ca7bd1398dbc started at 2022-11-29 23:39:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.349: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:41:09.349: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-6fhbq started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.349: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-df4qp started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:09.350: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-zqxgz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-mqv8r started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-dpvdl started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:09.350: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:09.350: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:09.350: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:09.350: INFO: hostpath-symlink-prep-provisioning-4482 started at 2022-11-29 23:39:52 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container init-volume-provisioning-4482 ready: false, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q54qk started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-pp6c2 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-gl5kt started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-sks6v started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.350: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container coredns ready: true, restart count 0 Nov 29 23:41:09.350: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-tj7m7 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.350: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.557: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.557: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:09.588: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 40183 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:09.588: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:09.620: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:09.653: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.653: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:41:09.653: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.653: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:41:09.653: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.653: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:41:09.653: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.653: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:41:09.653: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:09.653: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:09.653: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:09.653: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:09.653: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:09.653: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:09.654: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:09.654: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.654: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:41:09.654: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:41:09.654: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:41:09.654: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:41:09.654: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:41:09.654: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:09.654: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:09.654: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:41:09.654: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:41:09.654: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:41:09.654: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.654: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:41:09.654: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.654: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:41:09.827: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:09.827: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:09.857: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 44657 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:09.857: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:09.889: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-frcxh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-4ntql started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-msb99 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:09.947: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:09.947: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:09.947: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:09.947: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-5d2v5 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q5ddh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-c88hv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: hostexec-ip-172-20-40-36.us-east-2.compute.internal-wz66r started at 2022-11-29 23:41:07 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q6wnv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: ss2-2 started at 2022-11-29 23:40:10 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container webserver ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-2gvx9 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-9lv5f started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.947: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-hfc5n started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.947: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.197: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:10.198: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.227: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 43725 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:10.228: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.259: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-84mlt started at 2022-11-29 23:39:34 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-jzj9x started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container coredns ready: true, restart count 0 Nov 29 23:41:10.309: INFO: pod-subpath-test-preprovisionedpv-sqr9 started at 2022-11-29 23:40:46 +0000 UTC (1+2 container statuses recorded) Nov 29 23:41:10.309: INFO: Init container test-init-subpath-preprovisionedpv-sqr9 ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container test-container-subpath-preprovisionedpv-sqr9 ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container test-container-volume-preprovisionedpv-sqr9 ready: false, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-nflrz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:10.309: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.309: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.309: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-z9fkm started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: ss2-0 started at 2022-11-29 23:40:02 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container webserver ready: true, restart count 0 Nov 29 23:41:10.309: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:40:59 +0000 UTC (0+7 container statuses recorded) Nov 29 23:41:10.309: INFO: Container csi-attacher ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container csi-provisioner ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container csi-resizer ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container hostpath ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container liveness-probe ready: false, restart count 0 Nov 29 23:41:10.309: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-j28g9 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-4cwbg started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-brflb started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-f8pnz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:41:10.309: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:10.309: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:10.309: INFO: hostexec-ip-172-20-43-249.us-east-2.compute.internal-tgp22 started at 2022-11-29 23:40:29 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-hfhzd started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.309: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-snx6t started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.309: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.530: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.530: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.560: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 43726 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:36:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:10.560: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.592: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.636: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-jf6g2 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.636: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:10.636: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:10.636: INFO: startup-e09a04ba-dc4c-4e9d-b621-7d0fee069802 started at 2022-11-29 23:40:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Container busybox ready: false, restart count 0 Nov 29 23:41:10.636: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-g77zs started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.636: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:10.636: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.636: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.636: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:10.636: INFO: test-orphan-deployment-847dcfb7fb-nnl8d started at 2022-11-29 23:40:58 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Container httpd ready: false, restart count 0 Nov 29 23:41:10.636: INFO: ss2-1 started at 2022-11-29 23:40:06 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Container webserver ready: true, restart count 0 Nov 29 23:41:10.636: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-z4smf started at 2022-11-29 23:39:34 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.636: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.636: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-rtktn started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-88scb started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: rs-v6gzv started at 2022-11-29 23:39:47 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container donothing ready: false, restart count 0 Nov 29 23:41:10.637: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-dgdhs started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-sjbt7 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-d98vh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-5bbrm started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-7qqmv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.637: INFO: agnhost-primary-9vd6q started at 2022-11-29 23:40:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.637: INFO: Container agnhost-primary ready: false, restart count 0 Nov 29 23:41:10.829: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.829: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-9554" for this suite.
Find pod-f487d8c4-39ed-4b0e-a4ae-1e2d7c8b7a8a mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:41:09.446: Unexpected error: <*errors.errorString | 0xc003d733c0>: { s: "pod \"pod-fb934854-876a-471d-b486-4152eba45c0e\" is not Running: timed out waiting for the condition", } pod "pod-fb934854-876a-471d-b486-4152eba45c0e" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_02.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:36:09.081: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:36:09.230: INFO: Creating resource for dynamic PV Nov 29 23:36:09.230: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-8379-e2e-sch6ckd �[1mSTEP�[0m: creating a claim Nov 29 23:36:09.262: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-8379 with fsgroup 1000 Nov 29 23:41:09.446: FAIL: Unexpected error: <*errors.errorString | 0xc003d733c0>: { s: "pod \"pod-fb934854-876a-471d-b486-4152eba45c0e\" is not Running: timed out waiting for the condition", } pod "pod-fb934854-876a-471d-b486-4152eba45c0e" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc00448d340, 0xc001cad0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000827e00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000827e00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000827e00, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 29 23:41:09.476: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comhdfb7" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-8379". �[1mSTEP�[0m: Found 4 events. Nov 29 23:41:09.567: INFO: At 2022-11-29 23:36:09 +0000 UTC - event for ebs.csi.aws.comhdfb7: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:41:09.567: INFO: At 2022-11-29 23:36:09 +0000 UTC - event for ebs.csi.aws.comhdfb7: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-8379/ebs.csi.aws.comhdfb7" Nov 29 23:41:09.567: INFO: At 2022-11-29 23:36:09 +0000 UTC - event for ebs.csi.aws.comhdfb7: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:41:09.567: INFO: At 2022-11-29 23:36:19 +0000 UTC - event for ebs.csi.aws.comhdfb7: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8379-e2e-sch6ckd": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:41:09.596: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:41:09.596: INFO: pod-fb934854-876a-471d-b486-4152eba45c0e Pending [] Nov 29 23:41:09.596: INFO: Nov 29 23:41:09.626: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.655: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 43722 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:38:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{},"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:09.656: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.690: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.724: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.724: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:41:09.724: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-pp6c2 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.724: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.724: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.724: INFO: Container coredns ready: true, restart count 0 Nov 29 23:41:09.724: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-tj7m7 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.724: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-gl5kt started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-sks6v started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:09.725: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:09.725: INFO: liveness-a080bfd2-2aa6-4263-885d-ca7bd1398dbc started at 2022-11-29 23:39:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-6fhbq started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-df4qp started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-zqxgz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:41:09.725: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:09.725: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:09.725: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:09.725: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:09.725: INFO: hostpath-symlink-prep-provisioning-4482 started at 2022-11-29 23:39:52 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container init-volume-provisioning-4482 ready: false, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q54qk started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-mqv8r started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.725: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-dpvdl started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:09.725: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:09.914: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:41:09.914: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:09.945: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 40183 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:09.946: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:09.982: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:10.018: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:41:10.018: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:41:10.018: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:41:10.018: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:41:10.018: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:10.018: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:41:10.018: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:41:10.018: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.018: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:41:10.018: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:41:10.018: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:10.018: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.018: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:10.018: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:41:10.018: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.018: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:41:10.176: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:41:10.176: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:10.206: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 44657 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:10.207: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:10.239: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-5d2v5 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q5ddh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-msb99 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:10.273: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.273: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.273: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:10.273: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-c88hv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: hostexec-ip-172-20-40-36.us-east-2.compute.internal-wz66r started at 2022-11-29 23:41:07 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q6wnv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-2gvx9 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-9lv5f started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-hfc5n started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: ss2-2 started at 2022-11-29 23:40:10 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container webserver ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-frcxh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.273: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-4ntql started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.273: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.487: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:41:10.487: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.517: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 43725 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:10.518: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.549: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-hfhzd started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-snx6t started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-f8pnz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:41:10.585: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:10.585: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:10.585: INFO: hostexec-ip-172-20-43-249.us-east-2.compute.internal-tgp22 started at 2022-11-29 23:40:29 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-84mlt started at 2022-11-29 23:39:34 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-nflrz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-jzj9x started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container coredns ready: true, restart count 0 Nov 29 23:41:10.585: INFO: pod-subpath-test-preprovisionedpv-sqr9 started at 2022-11-29 23:40:46 +0000 UTC (1+2 container statuses recorded) Nov 29 23:41:10.585: INFO: Init container test-init-subpath-preprovisionedpv-sqr9 ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container test-container-subpath-preprovisionedpv-sqr9 ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container test-container-volume-preprovisionedpv-sqr9 ready: false, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-4cwbg started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-brflb started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:10.585: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.585: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.585: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-z9fkm started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.585: INFO: ss2-0 started at 2022-11-29 23:40:02 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container webserver ready: true, restart count 0 Nov 29 23:41:10.585: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:40:59 +0000 UTC (0+7 container statuses recorded) Nov 29 23:41:10.585: INFO: Container csi-attacher ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container csi-provisioner ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container csi-resizer ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container hostpath ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container liveness-probe ready: false, restart count 0 Nov 29 23:41:10.585: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 29 23:41:10.585: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-j28g9 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.585: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.760: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:41:10.760: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.790: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 43726 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:36:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:41:10.790: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.822: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:10.857: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:41:10.857: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:41:10.857: INFO: startup-e09a04ba-dc4c-4e9d-b621-7d0fee069802 started at 2022-11-29 23:40:50 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container busybox ready: false, restart count 0 Nov 29 23:41:10.857: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-g77zs started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.857: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-rtktn started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.857: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-88scb started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.857: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:41:10.857: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:41:10.857: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:41:10.857: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:41:10.857: INFO: test-orphan-deployment-847dcfb7fb-nnl8d started at 2022-11-29 23:40:58 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container httpd ready: false, restart count 0 Nov 29 23:41:10.857: INFO: ss2-1 started at 2022-11-29 23:40:06 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container webserver ready: true, restart count 0 Nov 29 23:41:10.857: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-z4smf started at 2022-11-29 23:39:34 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.857: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-7qqmv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.857: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.857: INFO: agnhost-primary-9vd6q started at 2022-11-29 23:40:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container agnhost-primary ready: false, restart count 0 Nov 29 23:41:10.858: INFO: rs-v6gzv started at 2022-11-29 23:39:47 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container donothing ready: false, restart count 0 Nov 29 23:41:10.858: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-dgdhs started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.858: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-sjbt7 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.858: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-d98vh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.858: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-5bbrm started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:10.858: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-jf6g2 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:41:10.858: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:41:11.032: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:41:11.032: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-8379" for this suite.
Find pod-fb934854-876a-471d-b486-4152eba45c0e mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sskips\sownership\schanges\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:32:00.462: Unexpected error: <*errors.errorString | 0xc00394cc30>: { s: "pod \"pod-5eb87b01-1b12-41a3-9af9-cbb5b9b6d4a9\" is not Running: timed out waiting for the condition", } pod "pod-5eb87b01-1b12-41a3-9af9-cbb5b9b6d4a9" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_15.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:27:00.081: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 29 23:27:00.228: INFO: Creating resource for dynamic PV Nov 29 23:27:00.228: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-8587-e2e-scw5sms �[1mSTEP�[0m: creating a claim Nov 29 23:27:00.261: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-8587 with fsgroup 1000 Nov 29 23:32:00.462: FAIL: Unexpected error: <*errors.errorString | 0xc00394cc30>: { s: "pod \"pod-5eb87b01-1b12-41a3-9af9-cbb5b9b6d4a9\" is not Running: timed out waiting for the condition", } pod "pod-5eb87b01-1b12-41a3-9af9-cbb5b9b6d4a9" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc003722000, 0xc004fdb0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00046fe00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00046fe00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00046fe00, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 29 23:32:00.494: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comwbzwm" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-8587". �[1mSTEP�[0m: Found 4 events. Nov 29 23:32:00.590: INFO: At 2022-11-29 23:27:00 +0000 UTC - event for ebs.csi.aws.comwbzwm: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:32:00.590: INFO: At 2022-11-29 23:27:00 +0000 UTC - event for ebs.csi.aws.comwbzwm: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:32:00.590: INFO: At 2022-11-29 23:27:00 +0000 UTC - event for ebs.csi.aws.comwbzwm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-8587/ebs.csi.aws.comwbzwm" Nov 29 23:32:00.590: INFO: At 2022-11-29 23:27:10 +0000 UTC - event for ebs.csi.aws.comwbzwm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8587-e2e-scw5sms": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:32:00.620: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:32:00.620: INFO: pod-5eb87b01-1b12-41a3-9af9-cbb5b9b6d4a9 Pending [] Nov 29 23:32:00.620: INFO: Nov 29 23:32:00.651: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:32:00.681: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 26881 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4453":"csi-mock-csi-mock-volumes-4453","ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-29 23:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-29 23:29:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:31:06 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:32:00.681: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:32:00.713: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:32:00.752: INFO: hostexec-ip-172-20-34-50.us-east-2.compute.internal-9ckvr started at 2022-11-29 23:31:53 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:00.753: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:32:00.753: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:00.753: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:32:00.753: INFO: csi-mockplugin-0 started at 2022-11-29 23:31:44 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:00.753: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:32:00.753: INFO: Container driver-registrar ready: true, restart count 0 Nov 29 23:32:00.753: INFO: Container mock ready: true, restart count 0 Nov 29 23:32:00.753: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:00.753: INFO: Container coredns ready: true, restart count 0 Nov 29 23:32:00.753: INFO: startup-script started at 2022-11-29 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:00.753: INFO: Container startup-script ready: true, restart count 0 Nov 29 23:32:00.753: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:32:00.753: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:32:00.753: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:32:00.753: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:00.753: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:32:00.753: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:00.753: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:32:00.753: INFO: rs-vgttn started at 2022-11-29 23:31:58 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:00.753: INFO: Container donothing ready: true, restart count 0 Nov 29 23:32:00.916: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:32:00.916: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:32:00.946: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 20634 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:27:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:32:00.947: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:32:00.979: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:32:01.041: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.041: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:32:01.041: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:01.041: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:32:01.041: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:01.041: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:32:01.041: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.041: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:32:01.041: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.042: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:32:01.042: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.042: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:32:01.042: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.042: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:32:01.042: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.042: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:32:01.042: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:32:01.042: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:32:01.042: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:32:01.042: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.042: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:32:01.042: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:32:01.042: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:32:01.042: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:32:01.042: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:32:01.042: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:32:01.042: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:01.042: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:32:01.042: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:32:01.042: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:32:01.278: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:32:01.278: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:32:01.309: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 26823 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:28:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:28:37 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:32:01.310: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:32:01.351: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:32:01.385: INFO: hostpath-symlink-prep-volume-4340 started at 2022-11-29 23:31:57 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container init-volume-volume-4340 ready: false, restart count 0 Nov 29 23:32:01.385: INFO: pod-submit-remove-86af3fa0-c474-4b04-9598-b1fe91b1e685 started at 2022-11-29 23:31:57 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container nginx ready: true, restart count 0 Nov 29 23:32:01.385: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:01.385: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:32:01.385: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:01.385: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:32:01.385: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:32:01.385: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:32:01.385: INFO: rs-gzl25 started at 2022-11-29 23:31:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container donothing ready: true, restart count 0 Nov 29 23:32:01.385: INFO: failed-jobs-history-limit-27829412-cx4qj started at 2022-11-29 23:32:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container c ready: false, restart count 0 Nov 29 23:32:01.385: INFO: pod-subpath-test-configmap-dn2z started at 2022-11-29 23:31:49 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container test-container-subpath-configmap-dn2z ready: true, restart count 0 Nov 29 23:32:01.385: INFO: rs-4kccb started at <nil> (0+0 container statuses recorded) Nov 29 23:32:01.385: INFO: hostexec-ip-172-20-40-36.us-east-2.compute.internal-r2lkz started at 2022-11-29 23:31:57 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:32:01.385: INFO: pod-ephm-test-projected-vxrn started at 2022-11-29 23:30:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.385: INFO: Container test-container-subpath-projected-vxrn ready: false, restart count 0 Nov 29 23:32:01.551: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:32:01.551: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:32:01.581: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 24907 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:30:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:30:07 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:32:01.582: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:32:01.615: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:32:01.652: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:32:01.652: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:32:01.652: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:32:01.652: INFO: pod-secrets-458971a2-cfa1-4a29-a534-a834593918ec started at 2022-11-29 23:30:25 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:01.652: INFO: Container creates-volume-test ready: true, restart count 0 Nov 29 23:32:01.652: INFO: Container dels-volume-test ready: true, restart count 0 Nov 29 23:32:01.652: INFO: Container upds-volume-test ready: true, restart count 0 Nov 29 23:32:01.652: INFO: rs-9bpbh started at 2022-11-29 23:30:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.652: INFO: Container donothing ready: false, restart count 0 Nov 29 23:32:01.652: INFO: rs-q9n5p started at 2022-11-29 23:31:54 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.652: INFO: Container donothing ready: true, restart count 0 Nov 29 23:32:01.652: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.652: INFO: Container coredns ready: true, restart count 0 Nov 29 23:32:01.652: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:01.652: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:32:01.652: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:01.652: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:32:01.866: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:32:01.866: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:32:01.896: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 27222 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-143":"ip-172-20-58-90.us-east-2.compute.internal","ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-29 23:31:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:31:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:59 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:59 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:31:59 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:31:59 +0000 UTC,LastTransitionTime:2022-11-29 23:17:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.90,},NodeAddress{Type:ExternalIP,Address:18.219.155.10,},NodeAddress{Type:Hostname,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-90.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-219-155-10.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f94aaad69954b6da688c5fa1cb4ee,SystemUUID:ec2f94aa-ad69-954b-6da6-88c5fa1cb4ee,BootID:b85f4ba4-0cee-4c63-b8a1-d5fd37dc064d,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-143^02189298-703e-11ed-b86f-7e98746e0dd1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-143^02189298-703e-11ed-b86f-7e98746e0dd1,DevicePath:,},},Config:nil,},} Nov 29 23:32:01.897: INFO: Logging kubelet events for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:32:01.934: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:32:01.969: INFO: cilium-l6j2h started at 2022-11-29 23:17:39 +0000 UTC (1+1 container statuses recorded) Nov 29 23:32:01.969: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:32:01.969: INFO: ebs-csi-node-7tqdj started at 2022-11-29 23:17:39 +0000 UTC (0+3 container statuses recorded) Nov 29 23:32:01.969: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:32:01.969: INFO: test-webserver-8f294c7e-7a26-4a12-a2d9-28c256c07264 started at 2022-11-29 23:28:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:32:01.969: INFO: Container test-webserver ready: true, restart count 0 Nov 29 23:32:01.969: INFO: pod-subpath-test-dynamicpv-jsbm started at 2022-11-29 23:31:56 +0000 UTC (2+2 container statuses recorded) Nov 29 23:32:01.969: INFO: Init container init-volume-dynamicpv-jsbm ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Init container test-init-subpath-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:32:01.969: INFO: Container test-container-subpath-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:32:01.969: INFO: Container test-container-volume-dynamicpv-jsbm ready: false, restart count 0 Nov 29 23:32:01.969: INFO: csi-hostpathplugin-0 started at 2022-11-29 23:31:52 +0000 UTC (0+7 container statuses recorded) Nov 29 23:32:01.969: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container hostpath ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:32:01.969: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:32:04.956: INFO: Latency metrics for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:32:04.956: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-8587" for this suite.
Find pod-5eb87b01-1b12-41a3-9af9-cbb5b9b6d4a9 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\smount\soptions$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 29 23:40:20.707: Unexpected error: <*errors.errorString | 0xc00431ac10>: { s: "pod \"pod-0e42cd13-ad02-498b-8f62-250c7a1dcd5b\" is not Running: timed out waiting for the condition", } pod "pod-0e42cd13-ad02-498b-8f62-250c7a1dcd5b" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 29 23:35:20.280: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should provision storage with mount options /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 29 23:35:20.433: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 23:35:20.433: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 23:35:20.433: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-1029-e2e-scxhvzf 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1029 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1029-e2e-scxhvzf,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1029 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1029-e2e-scxhvzf,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1029 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1029-e2e-scxhvzf,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-1029-e2e-scxhvzf 067447ae-1067-4c32-bb30-d6c3f4b96714 34181 0 2022-11-29 23:35:20 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-11-29 23:35:20 +0000 UTC FieldsV1 {"f:allowVolumeExpansion":{},"f:mountOptions":{},"f:parameters":{".":{},"f:encrypted":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:*Delete,MountOptions:[dirsync],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-k8bdm pvc- provisioning-1029 e0240905-d322-45da-9dfc-dcde3e80b3b4 34183 0 2022-11-29 23:35:20 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-11-29 23:35:20 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1029-e2e-scxhvzf,VolumeMode:*Filesystem,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} Nov 29 23:40:20.707: FAIL: Unexpected error: <*errors.errorString | 0xc00431ac10>: { s: "pod \"pod-0e42cd13-ad02-498b-8f62-250c7a1dcd5b\" is not Running: timed out waiting for the condition", } pod "pod-0e42cd13-ad02-498b-8f62-250c7a1dcd5b" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning(0x7973fe8, 0xc004631e40, 0xc0034b5340, 0xc004050000, 0xc004050a80, 0xc0040ec600, 0x0, 0x0, 0x0, 0x0, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418 +0xe7d k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:198 +0x245 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000975c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000975c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000975c80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 29 23:40:20.708: INFO: deleting claim "provisioning-1029"/"pvc-k8bdm" Nov 29 23:40:20.738: INFO: deleting storage class provisioning-1029-e2e-scxhvzf [AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "provisioning-1029". �[1mSTEP�[0m: Found 4 events. Nov 29 23:40:20.798: INFO: At 2022-11-29 23:35:20 +0000 UTC - event for pvc-k8bdm: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 29 23:40:20.798: INFO: At 2022-11-29 23:35:20 +0000 UTC - event for pvc-k8bdm: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 29 23:40:20.798: INFO: At 2022-11-29 23:35:20 +0000 UTC - event for pvc-k8bdm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } Provisioning: External provisioner is provisioning volume for claim "provisioning-1029/pvc-k8bdm" Nov 29 23:40:20.798: INFO: At 2022-11-29 23:35:30 +0000 UTC - event for pvc-k8bdm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-xwgrf_57eb266b-dc76-402c-b802-2a3c7fe01ee0 } ProvisioningFailed: failed to provision volume with StorageClass "provisioning-1029-e2e-scxhvzf": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:40:20.828: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:40:20.828: INFO: pod-0e42cd13-ad02-498b-8f62-250c7a1dcd5b Pending [] Nov 29 23:40:20.828: INFO: Nov 29 23:40:20.859: INFO: Logging node info for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:40:20.891: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-34-50.us-east-2.compute.internal 08308d51-84b9-4a2a-b0b0-ba63bdd6a38b 43722 0 2022-11-29 23:17:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-34-50.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-4221 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-34-50.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-009d86ea83e07db1f"} io.cilium.network.ipv4-cilium-host:172.20.32.59 io.cilium.network.ipv4-health-ip:172.20.50.86 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:38:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{},"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-009d86ea83e07db1f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:01 +0000 UTC,LastTransitionTime:2022-11-29 23:18:01 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:38:26 +0000 UTC,LastTransitionTime:2022-11-29 23:17:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.34.50,},NodeAddress{Type:ExternalIP,Address:18.222.162.252,},NodeAddress{Type:Hostname,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-34-50.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-222-162-252.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec250267feb8afaabbfdaf593403b141,SystemUUID:ec250267-feb8-afaa-bbfd-af593403b141,BootID:63a2a06a-789e-4b85-b7c1-e2d20d4625e9,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:40:20.891: INFO: Logging kubelet events for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:40:20.923: INFO: Logging pods the kubelet thinks is on node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:40:20.959: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-dpvdl started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: ebs-csi-node-s2r5j started at 2022-11-29 23:17:36 +0000 UTC (0+3 container statuses recorded) Nov 29 23:40:20.960: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:40:20.960: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:40:20.960: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:40:20.960: INFO: hostpath-symlink-prep-provisioning-4482 started at 2022-11-29 23:39:52 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container init-volume-provisioning-4482 ready: false, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q54qk started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-mqv8r started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: coredns-autoscaler-6658b4bf85-ckxg7 started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container autoscaler ready: true, restart count 0 Nov 29 23:40:20.960: INFO: test-webserver-da0ebd18-e4c2-41f3-9735-a5b9aced80bb started at 2022-11-29 23:39:27 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container test-webserver ready: false, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-pp6c2 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-sks6v started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: coredns-5fcc7b6498-24lvd started at 2022-11-29 23:17:55 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container coredns ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-tj7m7 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-gl5kt started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-6fhbq started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-df4qp started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cilium-2pmw9 started at 2022-11-29 23:17:36 +0000 UTC (1+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:40:20.960: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:40:20.960: INFO: liveness-a080bfd2-2aa6-4263-885d-ca7bd1398dbc started at 2022-11-29 23:39:01 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container agnhost-container ready: true, restart count 0 Nov 29 23:40:20.960: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-zqxgz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:20.960: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:40:21.145: INFO: Latency metrics for node ip-172-20-34-50.us-east-2.compute.internal Nov 29 23:40:21.145: INFO: Logging node info for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:40:21.177: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-35-195.us-east-2.compute.internal 400419b1-7711-49f5-b581-a2ce8acd8fa1 40183 0 2022-11-29 23:15:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-35-195.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08639b177769f6d4e"} io.cilium.network.ipv4-cilium-host:172.20.38.159 io.cilium.network.ipv4-health-ip:172.20.62.43 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 23:15:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-29 23:15:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-29 23:16:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-29 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:17:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08639b177769f6d4e,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:16:59 +0000 UTC,LastTransitionTime:2022-11-29 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:15:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:37:46 +0000 UTC,LastTransitionTime:2022-11-29 23:16:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.195,},NodeAddress{Type:ExternalIP,Address:18.191.233.79,},NodeAddress{Type:Hostname,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-35-195.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-191-233-79.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a682f701ad531c829b3ed13978c2b,SystemUUID:ec2a682f-701a-d531-c829-b3ed13978c2b,BootID:af01f3b3-f90b-4396-b0e1-57d3bd93314b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42905758,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441806,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955305,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:40:21.177: INFO: Logging kubelet events for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:40:21.210: INFO: Logging pods the kubelet thinks is on node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:40:21.244: INFO: ebs-csi-node-6sd8m started at 2022-11-29 23:16:00 +0000 UTC (0+3 container statuses recorded) Nov 29 23:40:21.244: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:40:21.244: INFO: kops-controller-5hp2r started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container kops-controller ready: true, restart count 0 Nov 29 23:40:21.244: INFO: cilium-operator-587dd66f87-zr66z started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container cilium-operator ready: true, restart count 0 Nov 29 23:40:21.244: INFO: kube-scheduler-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container kube-scheduler ready: true, restart count 0 Nov 29 23:40:21.244: INFO: etcd-manager-events-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:40:21.244: INFO: etcd-manager-main-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container etcd-manager ready: true, restart count 0 Nov 29 23:40:21.244: INFO: cilium-fjsqn started at 2022-11-29 23:16:00 +0000 UTC (1+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:40:21.244: INFO: dns-controller-5fb778d9d5-hj564 started at 2022-11-29 23:16:00 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container dns-controller ready: true, restart count 0 Nov 29 23:40:21.244: INFO: ebs-csi-controller-764b64d679-xwgrf started at 2022-11-29 23:16:00 +0000 UTC (0+5 container statuses recorded) Nov 29 23:40:21.244: INFO: Container csi-attacher ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container csi-provisioner ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container csi-resizer ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:40:21.244: INFO: kube-apiserver-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+2 container statuses recorded) Nov 29 23:40:21.244: INFO: Container healthcheck ready: true, restart count 0 Nov 29 23:40:21.244: INFO: Container kube-apiserver ready: true, restart count 1 Nov 29 23:40:21.244: INFO: kube-controller-manager-ip-172-20-35-195.us-east-2.compute.internal started at 2022-11-29 23:14:45 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.244: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 29 23:40:21.423: INFO: Latency metrics for node ip-172-20-35-195.us-east-2.compute.internal Nov 29 23:40:21.423: INFO: Logging node info for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:40:21.453: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-40-36.us-east-2.compute.internal af993eb1-0e92-4f73-b790-f9bd3cc5d982 44657 0 2022-11-29 23:17:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-40-36.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-40-36.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdc28aa89d8465d9"} io.cilium.network.ipv4-cilium-host:172.20.62.239 io.cilium.network.ipv4-health-ip:172.20.58.129 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0cdc28aa89d8465d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:03 +0000 UTC,LastTransitionTime:2022-11-29 23:18:03 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:39:28 +0000 UTC,LastTransitionTime:2022-11-29 23:17:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.36,},NodeAddress{Type:ExternalIP,Address:3.138.60.3,},NodeAddress{Type:Hostname,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-40-36.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-138-60-3.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29b3b1f9cfa134fc8d27cd81bc0d9e,SystemUUID:ec29b3b1-f9cf-a134-fc8d-27cd81bc0d9e,BootID:3f3a5b13-b8cb-408a-97be-590267d2fcf7,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:40:21.453: INFO: Logging kubelet events for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:40:21.485: INFO: Logging pods the kubelet thinks is on node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:40:21.519: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-hfc5n started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.519: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.519: INFO: ss2-2 started at 2022-11-29 23:40:10 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.519: INFO: Container webserver ready: false, restart count 0 Nov 29 23:40:21.519: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-2gvx9 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.519: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.519: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-9lv5f started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.519: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.519: INFO: all-pods-removed-fbvj6 started at 2022-11-29 23:40:04 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.519: INFO: Container c ready: true, restart count 0 Nov 29 23:40:21.519: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-frcxh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.519: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.519: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-4ntql started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.520: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-5d2v5 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.520: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q5ddh started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.520: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-msb99 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.520: INFO: ebs-csi-node-zh5pc started at 2022-11-29 23:17:38 +0000 UTC (0+3 container statuses recorded) Nov 29 23:40:21.520: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:40:21.520: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:40:21.520: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:40:21.520: INFO: cilium-nf5r9 started at 2022-11-29 23:17:38 +0000 UTC (1+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:40:21.520: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:40:21.520: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-c88hv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.520: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-q6wnv started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.520: INFO: busybox-c3945811-0a68-45e3-9fd9-3b165f5496fd started at 2022-11-29 23:36:21 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.520: INFO: Container busybox ready: true, restart count 0 Nov 29 23:40:21.721: INFO: Latency metrics for node ip-172-20-40-36.us-east-2.compute.internal Nov 29 23:40:21.721: INFO: Logging node info for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:40:21.751: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-249.us-east-2.compute.internal 5ac0d0ea-bf53-4b61-ad3c-487f4fffc78f 43725 0 2022-11-29 23:17:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-249.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-249.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-031255ff47b29623a"} io.cilium.network.ipv4-cilium-host:172.20.63.97 io.cilium.network.ipv4-health-ip:172.20.44.68 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:17:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:39:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-031255ff47b29623a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:02 +0000 UTC,LastTransitionTime:2022-11-29 23:18:02 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:39:17 +0000 UTC,LastTransitionTime:2022-11-29 23:17:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.249,},NodeAddress{Type:ExternalIP,Address:3.145.33.126,},NodeAddress{Type:Hostname,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-249.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-33-126.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d491bfd6e4c9de7bc5abecf5220f1,SystemUUID:ec2d491b-fd6e-4c9d-e7bc-5abecf5220f1,BootID:1779f3d5-79da-4fa1-acf1-e125adb54dec,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:40:21.751: INFO: Logging kubelet events for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:40:21.784: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:40:21.818: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-84mlt started at 2022-11-29 23:39:34 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: coredns-5fcc7b6498-z8ghf started at 2022-11-29 23:18:17 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container coredns ready: true, restart count 0 Nov 29 23:40:21.819: INFO: pod-terminate-status-0-4 started at 2022-11-29 23:40:19 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container fail ready: false, restart count 0 Nov 29 23:40:21.819: INFO: busybox-015c8984-b9b0-4129-aca0-0823b626a0d7 started at 2022-11-29 23:40:14 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container busybox ready: true, restart count 0 Nov 29 23:40:21.819: INFO: pod-terminate-status-1-4 started at <nil> (0+0 container statuses recorded) Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-nflrz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-jzj9x started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: ebs-csi-node-msn9b started at 2022-11-29 23:17:37 +0000 UTC (0+3 container statuses recorded) Nov 29 23:40:21.819: INFO: Container ebs-plugin ready: true, restart count 0 Nov 29 23:40:21.819: INFO: Container liveness-probe ready: true, restart count 0 Nov 29 23:40:21.819: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-z9fkm started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: ss2-0 started at 2022-11-29 23:40:02 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container webserver ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-j28g9 started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-4cwbg started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-brflb started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cilium-2qfpl started at 2022-11-29 23:17:37 +0000 UTC (1+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 29 23:40:21.819: INFO: Container cilium-agent ready: true, restart count 0 Nov 29 23:40:21.819: INFO: pod-terminate-status-2-5 started at <nil> (0+0 container statuses recorded) Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-hfhzd started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-snx6t started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: true, restart count 0 Nov 29 23:40:21.819: INFO: cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796-f8pnz started at 2022-11-29 23:39:35 +0000 UTC (0+1 container statuses recorded) Nov 29 23:40:21.819: INFO: Container cleanup40-bea864c6-5c59-4b57-a592-12f8d8b91796 ready: false, restart count 0 Nov 29 23:40:22.026: INFO: Latency metrics for node ip-172-20-43-249.us-east-2.compute.internal Nov 29 23:40:22.026: INFO: Logging node info for node ip-172-20-58-90.us-east-2.compute.internal Nov 29 23:40:22.056: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-90.us-east-2.compute.internal f5d39a7c-b668-41dc-af7a-be22c076af86 43726 0 2022-11-29 23:17:38 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-90.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-58-90.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02543b1ae3251ed53"} io.cilium.network.ipv4-cilium-host:172.20.58.162 io.cilium.network.ipv4-health-ip:172.20.46.7 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-29 23:17:38 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-29 23:18:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-29 23:18:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-29 23:36:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-29 23:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-02543b1ae3251ed53,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 23:18:07 +0000 UTC,LastTransitionTime:2022-11-29 23:18:07 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UTC,LastTransitionTime:2022-11-29 23:17:38 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:37:10 +0000 UT