Result | FAILURE |
Tests | 53 failed / 713 succeeded |
Started | |
Elapsed | 53m5s |
Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 24 23:48:21.469: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc005a1afd0>: { s: "pod \"pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f\" is not Running: timed out waiting for the condition", } pod "pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_20.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:43:21.104: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 24 23:43:21.254: INFO: Creating resource for dynamic PV Nov 24 23:43:21.254: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-735-e2e-scb84kk �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 24 23:48:21.469: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc005a1afd0>: { s: "pod \"pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f\" is not Running: timed out waiting for the condition", } pod "pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00093c300) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00093c300) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00093c300, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 24 23:48:21.470: INFO: Deleting pod "pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f" in namespace "volume-expand-735" Nov 24 23:48:21.504: INFO: Wait up to 5m0s for pod "pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 24 23:48:21.534: INFO: Deleting pod "pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f" in namespace "volume-expand-735" �[1mSTEP�[0m: Deleting pvc Nov 24 23:48:21.593: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comjpdp6" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-735". �[1mSTEP�[0m: Found 5 events. Nov 24 23:48:21.684: INFO: At 2022-11-24 23:43:21 +0000 UTC - event for ebs.csi.aws.comjpdp6: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:48:21.685: INFO: At 2022-11-24 23:43:21 +0000 UTC - event for ebs.csi.aws.comjpdp6: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:48:21.685: INFO: At 2022-11-24 23:43:21 +0000 UTC - event for ebs.csi.aws.comjpdp6: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-735/ebs.csi.aws.comjpdp6" Nov 24 23:48:21.685: INFO: At 2022-11-24 23:43:31 +0000 UTC - event for ebs.csi.aws.comjpdp6: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-735-e2e-scb84kk": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:48:21.685: INFO: At 2022-11-24 23:43:54 +0000 UTC - event for ebs.csi.aws.comjpdp6: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-735-e2e-scb84kk": rpc error: code = Internal desc = Could not create volume "pvc-1f98d5fa-455f-49ba-9218-90cb1bae9d64": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 24 23:48:21.714: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:48:21.714: INFO: Nov 24 23:48:21.745: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:48:21.775: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 48303 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:48:21.775: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:48:21.807: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:48:21.869: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:48:21.869: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:48:21.869: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:48:21.869: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:48:21.869: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:48:21.869: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:48:21.869: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:48:21.869: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:48:21.869: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:48:21.869: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:48:21.869: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:48:21.869: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:48:21.869: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:48:21.869: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:48:21.869: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:21.869: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:48:22.026: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:48:22.026: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:48:22.056: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 46971 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:45:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:46:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:46:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:46:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:46:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:48:22.056: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:48:22.089: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:48:22.121: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:48:22.122: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:48:22.122: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:48:22.122: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:48:22.122: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:48:22.122: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:48:22.122: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:48:22.296: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:48:22.296: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:48:22.327: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 44021 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:48:22.327: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:48:22.361: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:48:22.395: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:48:22.395: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:48:22.395: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:48:22.395: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:48:22.395: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:48:22.395: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:48:22.395: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:48:22.395: INFO: webserver-pod started at 2022-11-24 23:45:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:22.395: INFO: Container agnhost ready: false, restart count 0 Nov 24 23:48:22.556: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:48:22.556: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:48:22.586: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 47052 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-24 23:45:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:46:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:46:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:46:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:46:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:48:22.587: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:48:22.619: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:48:22.651: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:48:22.651: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:48:22.651: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:48:22.651: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:22.651: INFO: Container coredns ready: true, restart count 0 Nov 24 23:48:22.651: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:48:22.651: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:48:22.651: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:48:22.651: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:48:22.803: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:48:22.804: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:48:22.834: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 47661 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:45:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:48:22.834: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:48:22.870: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:48:22.902: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:22.902: INFO: Container coredns ready: true, restart count 0 Nov 24 23:48:22.902: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:48:22.902: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:48:22.902: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:48:22.902: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:48:22.902: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:48:22.902: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:48:22.902: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:48:22.902: INFO: test-pod started at 2022-11-24 23:46:31 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:22.902: INFO: Container test-container ready: false, restart count 0 Nov 24 23:48:22.902: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:48:22.902: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:48:23.063: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:48:23.063: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-735" for this suite.
Find pod-b4f5b4c2-6600-4045-86a6-3fd2ddf2188f mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 24 23:51:30.956: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc00459ae60>: { s: "pod \"pod-514cf656-caed-4c4e-9bdc-e25627a2e20f\" is not Running: timed out waiting for the condition", } pod "pod-514cf656-caed-4c4e-9bdc-e25627a2e20f" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_17.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:46:30.590: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 24 23:46:30.741: INFO: Creating resource for dynamic PV Nov 24 23:46:30.741: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-8796-e2e-scdd72v �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 24 23:51:30.955: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc00459ae60>: { s: "pod \"pod-514cf656-caed-4c4e-9bdc-e25627a2e20f\" is not Running: timed out waiting for the condition", } pod "pod-514cf656-caed-4c4e-9bdc-e25627a2e20f" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001a7500) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0001a7500) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0001a7500, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 24 23:51:30.956: INFO: Deleting pod "pod-514cf656-caed-4c4e-9bdc-e25627a2e20f" in namespace "volume-expand-8796" Nov 24 23:51:30.988: INFO: Wait up to 5m0s for pod "pod-514cf656-caed-4c4e-9bdc-e25627a2e20f" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 24 23:51:31.018: INFO: Deleting pod "pod-514cf656-caed-4c4e-9bdc-e25627a2e20f" in namespace "volume-expand-8796" �[1mSTEP�[0m: Deleting pvc Nov 24 23:51:31.078: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comqjkpg" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-8796". �[1mSTEP�[0m: Found 4 events. Nov 24 23:51:31.169: INFO: At 2022-11-24 23:46:30 +0000 UTC - event for ebs.csi.aws.comqjkpg: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:51:31.169: INFO: At 2022-11-24 23:46:30 +0000 UTC - event for ebs.csi.aws.comqjkpg: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:51:31.169: INFO: At 2022-11-24 23:46:30 +0000 UTC - event for ebs.csi.aws.comqjkpg: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-8796/ebs.csi.aws.comqjkpg" Nov 24 23:51:31.169: INFO: At 2022-11-24 23:46:40 +0000 UTC - event for ebs.csi.aws.comqjkpg: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8796-e2e-scdd72v": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:51:31.198: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:51:31.198: INFO: Nov 24 23:51:31.228: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:51:31.258: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 48303 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:47:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:51:31.258: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:51:31.290: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:51:31.324: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.324: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:51:31.324: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:51:31.324: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:51:31.324: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.324: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:51:31.324: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:51:31.324: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:51:31.324: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:51:31.324: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:51:31.324: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:51:31.324: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.324: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:51:31.324: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.325: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:51:31.325: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:51:31.325: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:51:31.325: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:51:31.325: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.325: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:51:31.325: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.325: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:51:31.325: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:31.325: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:51:31.485: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:51:31.485: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:51:31.515: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 48993 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:45:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:51:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:51:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:51:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:51:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:51:31.516: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:51:31.547: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:51:31.579: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:51:31.579: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:51:31.579: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:51:31.579: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:51:31.579: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:51:31.579: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:51:31.579: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:51:31.765: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:51:31.765: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:51:31.795: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 48700 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:49:38 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:49:38 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:49:38 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:49:38 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:51:31.795: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:51:31.850: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:51:31.917: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:51:31.917: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:51:31.917: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:51:31.917: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:51:31.917: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:51:31.917: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:51:31.917: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:51:32.075: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:51:32.075: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:51:32.105: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 49003 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-24 23:45:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:51:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:51:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:51:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:51:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:51:32.105: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:51:32.137: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:51:32.169: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:51:32.169: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:51:32.169: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:51:32.169: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:32.169: INFO: Container coredns ready: true, restart count 0 Nov 24 23:51:32.169: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:51:32.169: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:51:32.169: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:51:32.169: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:51:32.323: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:51:32.323: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:51:32.353: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 48957 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:45:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:50:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:50:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:50:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:50:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:51:32.353: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:51:32.385: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:51:32.417: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:32.417: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:51:32.417: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:51:32.417: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:51:32.417: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:51:32.417: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:51:32.418: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:51:32.418: INFO: Container coredns ready: true, restart count 0 Nov 24 23:51:32.418: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:51:32.418: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:51:32.418: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:51:32.581: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:51:32.582: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-8796" for this suite.
Find pod-514cf656-caed-4c4e-9bdc-e25627a2e20f mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 24 23:32:35.826: Unexpected error: <*errors.errorString | 0xc0002ba280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385from junit_08.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:27:35.470: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volumemode �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should not mount / map unused volumes in a pod [LinuxOnly] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 24 23:27:35.616: INFO: Creating resource for dynamic PV Nov 24 23:27:35.616: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volumemode-9578-e2e-scd5ps9 �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating pod Nov 24 23:32:35.826: FAIL: Unexpected error: <*errors.errorString | 0xc0002ba280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385 +0x6c5 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0008ba900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0008ba900) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0008ba900, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 24 23:32:35.826: INFO: Deleting pod "pod-82b23325-894f-4c2f-aa6a-0c4e61a76ace" in namespace "volumemode-9578" Nov 24 23:32:35.860: INFO: Wait up to 5m0s for pod "pod-82b23325-894f-4c2f-aa6a-0c4e61a76ace" to be fully deleted �[1mSTEP�[0m: Deleting pvc Nov 24 23:32:35.917: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.combzkqn" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volumemode-9578". �[1mSTEP�[0m: Found 5 events. Nov 24 23:32:36.008: INFO: At 2022-11-24 23:27:35 +0000 UTC - event for ebs.csi.aws.combzkqn: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:32:36.008: INFO: At 2022-11-24 23:27:35 +0000 UTC - event for ebs.csi.aws.combzkqn: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:32:36.008: INFO: At 2022-11-24 23:27:35 +0000 UTC - event for ebs.csi.aws.combzkqn: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "volumemode-9578/ebs.csi.aws.combzkqn" Nov 24 23:32:36.008: INFO: At 2022-11-24 23:27:45 +0000 UTC - event for ebs.csi.aws.combzkqn: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-9578-e2e-scd5ps9": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:32:36.008: INFO: At 2022-11-24 23:29:06 +0000 UTC - event for ebs.csi.aws.combzkqn: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-9578-e2e-scd5ps9": rpc error: code = Internal desc = Could not create volume "pvc-e9fb5138-6edb-4210-835b-0ea270f129a6": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 24 23:32:36.037: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:32:36.037: INFO: Nov 24 23:32:36.068: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:32:36.098: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 18329 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:32:36.098: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:32:36.130: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:32:36.168: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:32:36.169: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:32:36.169: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:32:36.169: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:36.169: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:32:36.169: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:32:36.169: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:32:36.169: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:36.169: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:32:36.169: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:32:36.169: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:32:36.169: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:32:36.169: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:32:36.169: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:32:36.169: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.169: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:32:36.339: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:32:36.340: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:32:36.369: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 23013 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-24 23:29:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:32:36.370: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:32:36.401: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-d6lg5 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: deployment-55649fd747-m4s8g started at 2022-11-24 23:30:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container nginx ready: false, restart count 0 Nov 24 23:32:36.436: INFO: all-succeed-fb8tg started at 2022-11-24 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container c ready: false, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-vzj49 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-x7svl started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:32:36.436: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:32:36.436: INFO: pod-c39c6f03-f30b-48b1-95de-9abc2de560b5 started at 2022-11-24 23:32:11 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container test-container ready: false, restart count 0 Nov 24 23:32:36.436: INFO: ss2-0 started at 2022-11-24 23:29:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container webserver ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-2fpvr started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: deployment-585449566-2k7fg started at 2022-11-24 23:30:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container nginx ready: false, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-bdp96 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-nrsr8 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: affinity-clusterip-transition-h57dd started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-lbxg8 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-b8skk started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:36.436: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:32:36.436: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:36.436: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:32:36.436: INFO: pod-ephm-test-projected-vmdh started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container test-container-subpath-projected-vmdh ready: false, restart count 0 Nov 24 23:32:36.436: INFO: all-succeed-84gkf started at 2022-11-24 23:30:56 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container c ready: false, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-7nj8h started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.436: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-jkd2x started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.436: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.741: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:32:36.741: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:32:36.770: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 24030 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-8418":"ip-172-20-43-101.us-east-2.compute.internal","ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:31:00 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-8418^0c25b5d4-6c50-11ed-964c-6aeca698b218],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-8418^0c25b5d4-6c50-11ed-964c-6aeca698b218,DevicePath:,},},Config:nil,},} Nov 24 23:32:36.771: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:32:36.803: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:32:36.842: INFO: terminate-cmd-rpa3dea59ed-50f8-45f3-9058-9f2be1549f23 started at 2022-11-24 23:31:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container terminate-cmd-rpa ready: false, restart count 0 Nov 24 23:32:36.842: INFO: pfpod started at 2022-11-24 23:31:44 +0000 UTC (0+2 container statuses recorded) Nov 24 23:32:36.842: INFO: Container portforwardtester ready: false, restart count 0 Nov 24 23:32:36.842: INFO: Container readiness ready: false, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-kpnff started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-ct29k started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6bw2w started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-r6lww started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-7gn8v started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-nvkzw started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.842: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.842: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-42q46 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.843: INFO: all-succeed-pbfv2 started at 2022-11-24 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container c ready: false, restart count 0 Nov 24 23:32:36.843: INFO: all-succeed-crhdn started at 2022-11-24 23:30:55 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container c ready: false, restart count 0 Nov 24 23:32:36.843: INFO: ss2-1 started at 2022-11-24 23:31:49 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container webserver ready: false, restart count 0 Nov 24 23:32:36.843: INFO: liveness-40c23abd-9995-4576-80a7-816fd92cf233 started at 2022-11-24 23:28:49 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:32:36.843: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-k4cjb started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.843: INFO: httpd started at 2022-11-24 23:29:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container httpd ready: false, restart count 0 Nov 24 23:32:36.843: INFO: pod-subpath-test-dynamicpv-h588 started at 2022-11-24 23:30:59 +0000 UTC (2+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Init container init-volume-dynamicpv-h588 ready: false, restart count 0 Nov 24 23:32:36.843: INFO: Init container test-init-volume-dynamicpv-h588 ready: false, restart count 0 Nov 24 23:32:36.843: INFO: Container test-container-subpath-dynamicpv-h588 ready: false, restart count 0 Nov 24 23:32:36.843: INFO: affinity-clusterip-transition-pqdk4 started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container affinity-clusterip-transition ready: false, restart count 0 Nov 24 23:32:36.843: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-2hsxc started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.843: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:32:36.843: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:29:58 +0000 UTC (0+7 container statuses recorded) Nov 24 23:32:36.843: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:32:36.843: INFO: ss-0 started at 2022-11-24 23:30:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container webserver ready: false, restart count 0 Nov 24 23:32:36.843: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:36.843: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:36.843: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:32:36.843: INFO: deployment-585449566-djvp7 started at 2022-11-24 23:30:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container nginx ready: false, restart count 0 Nov 24 23:32:36.843: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-fbmch started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:36.843: INFO: my-hostname-basic-e7bd73af-a28d-41ad-9c02-a01481e6ab23-9wfgf started at 2022-11-24 23:30:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:36.843: INFO: Container my-hostname-basic-e7bd73af-a28d-41ad-9c02-a01481e6ab23 ready: false, restart count 0 Nov 24 23:32:37.060: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:32:37.060: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:32:37.089: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 21783 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:21:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{},"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:32:37.090: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:32:37.122: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-lqfts started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-b876n started at 2022-11-24 23:30:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:32:37.157: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container coredns ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-j7d5q started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-4ljg9 started at 2022-11-24 23:30:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:32:37.157: INFO: pod-5a5880cd-3940-42c2-9ae9-640bf68eebcd started at 2022-11-24 23:31:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container write-pod ready: false, restart count 0 Nov 24 23:32:37.157: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:32:37.157: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:32:37.157: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:30:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container csi-attacher ready: false, restart count 0 Nov 24 23:32:37.157: INFO: affinity-clusterip-transition-67vl4 started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6hj9d started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: pod-subpath-test-preprovisionedpv-nm7j started at 2022-11-24 23:31:05 +0000 UTC (1+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Init container init-volume-preprovisionedpv-nm7j ready: false, restart count 0 Nov 24 23:32:37.157: INFO: Container test-container-subpath-preprovisionedpv-nm7j ready: false, restart count 0 Nov 24 23:32:37.157: INFO: csi-mockplugin-0 started at 2022-11-24 23:30:46 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:37.157: INFO: Container csi-provisioner ready: false, restart count 0 Nov 24 23:32:37.157: INFO: Container driver-registrar ready: false, restart count 0 Nov 24 23:32:37.157: INFO: Container mock ready: false, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6xwrp started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:37.157: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:32:37.157: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:37.157: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-frgmv started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-5nzcw started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-dmtrc started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-xmxdf started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-qvzq5 started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.157: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-69l56 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.157: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.332: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:32:37.332: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:32:37.361: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 24089 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9504":"csi-mock-csi-mock-volumes-9504","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-24 23:31:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9504^4],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:32:37.362: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:32:37.397: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:32:37.432: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-29zhg started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: false, restart count 0 Nov 24 23:32:37.432: INFO: deployment-585449566-rw9ss started at 2022-11-24 23:30:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container nginx ready: false, restart count 0 Nov 24 23:32:37.432: INFO: pvc-volume-tester-l444q started at 2022-11-24 23:31:09 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container volume-tester ready: false, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-5ljph started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: hostexec-ip-172-20-63-204.us-east-2.compute.internal-cfq8w started at 2022-11-24 23:31:34 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-tml7d started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-tfmhr started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: csi-mockplugin-0 started at 2022-11-24 23:29:58 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:37.432: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:32:37.432: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:32:37.432: INFO: Container mock ready: true, restart count 0 Nov 24 23:32:37.432: INFO: pod-subpath-test-preprovisionedpv-cv7f started at 2022-11-24 23:31:51 +0000 UTC (1+2 container statuses recorded) Nov 24 23:32:37.432: INFO: Init container test-init-subpath-preprovisionedpv-cv7f ready: false, restart count 0 Nov 24 23:32:37.432: INFO: Container test-container-subpath-preprovisionedpv-cv7f ready: false, restart count 0 Nov 24 23:32:37.432: INFO: Container test-container-volume-preprovisionedpv-cv7f ready: false, restart count 0 Nov 24 23:32:37.432: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:29:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container csi-attacher ready: false, restart count 0 Nov 24 23:32:37.432: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:32:37.432: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:32:37.432: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:32:37.432: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:32:37.432: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container coredns ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-ntd7d started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-n994f started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-7ctz6 started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:32:37.432: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-xt4tv started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6sxsd started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.432: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-g4dhj started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:32:37.432: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:32:37.604: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:32:37.604: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volumemode-9578" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumes\sshould\sstore\sdata$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 24 23:26:49.072: Failed to create injector pod: timed out waiting for the condition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186from junit_07.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:21:48.607: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should store data /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 24 23:21:48.766: INFO: Creating resource for dynamic PV Nov 24 23:21:48.766: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volume-5307-e2e-scnlfmg �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: starting external-injector �[1mSTEP�[0m: Deleting pod external-injector in namespace volume-5307 Nov 24 23:26:49.038: INFO: Waiting for pod external-injector to disappear Nov 24 23:26:49.072: INFO: Pod external-injector no longer exists Nov 24 23:26:49.072: FAIL: Failed to create injector pod: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumesTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186 +0x3ff k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0007c4780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0007c4780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0007c4780, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: cleaning the environment after external �[1mSTEP�[0m: Deleting pvc Nov 24 23:26:49.103: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comjmvf9" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-5307". �[1mSTEP�[0m: Found 4 events. Nov 24 23:26:49.204: INFO: At 2022-11-24 23:21:48 +0000 UTC - event for ebs.csi.aws.comjmvf9: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:26:49.204: INFO: At 2022-11-24 23:21:48 +0000 UTC - event for ebs.csi.aws.comjmvf9: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:26:49.204: INFO: At 2022-11-24 23:21:48 +0000 UTC - event for ebs.csi.aws.comjmvf9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "volume-5307/ebs.csi.aws.comjmvf9" Nov 24 23:26:49.204: INFO: At 2022-11-24 23:21:58 +0000 UTC - event for ebs.csi.aws.comjmvf9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-5307-e2e-scnlfmg": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:26:49.237: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:26:49.237: INFO: Nov 24 23:26:49.269: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:49.300: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 7955 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:49.300: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:49.338: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:49.412: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:26:49.412: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:49.412: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:26:49.412: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:26:49.412: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:26:49.412: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:49.412: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:49.412: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:26:49.412: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:26:49.412: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:26:49.412: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:26:49.412: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:26:49.412: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:26:49.412: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:49.412: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:49.412: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:26:49.596: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:49.596: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:49.627: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 14177 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:22:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:49.627: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:49.663: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:49.697: INFO: pod-projected-configmaps-1cbd4a7c-7ed0-4a48-8b65-90d6489d8c89 started at 2022-11-24 23:25:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:49.697: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:49.697: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:49.697: INFO: send-events-1823ab19-7762-4837-a313-917ad3764286 started at 2022-11-24 23:26:49 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container p ready: false, restart count 0 Nov 24 23:26:49.697: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:49.697: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:49.697: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:49.697: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:49.697: INFO: netserver-0 started at 2022-11-24 23:26:34 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:49.697: INFO: netserver-0 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:49.697: INFO: pod-exec-websocket-855996c2-1be2-439b-b2c2-d9db0f99c6b6 started at 2022-11-24 23:26:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container main ready: true, restart count 0 Nov 24 23:26:49.697: INFO: svc-latency-rc-8fmcj started at 2022-11-24 23:26:43 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container svc-latency-rc ready: true, restart count 0 Nov 24 23:26:49.697: INFO: condition-test-tdwqk started at 2022-11-24 23:26:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container httpd ready: false, restart count 0 Nov 24 23:26:49.697: INFO: busybox-ff078305-a49e-4a13-9ecb-ea964e7b4f5e started at 2022-11-24 23:26:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.697: INFO: Container busybox ready: true, restart count 0 Nov 24 23:26:49.697: INFO: pod-configmaps-0b5e3ba7-c6f0-4842-8f57-d30e0c815328 started at 2022-11-24 23:26:41 +0000 UTC (0+2 container statuses recorded) Nov 24 23:26:49.697: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:49.697: INFO: Container configmap-volume-binary-test ready: false, restart count 0 Nov 24 23:26:49.855: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:49.855: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:49.886: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 14794 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6460":"ip-172-20-43-101.us-east-2.compute.internal","ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-11-24 23:26:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6460^758025c2-6c4f-11ed-bebc-2a2dfedf8700,DevicePath:,},},Config:nil,},} Nov 24 23:26:49.887: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:49.925: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:49.965: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:49.965: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:49.965: INFO: inline-volume-tester-9fn6r started at 2022-11-24 23:26:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.965: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 23:26:49.965: INFO: busybox-c6872ff6-28c5-46f4-bd84-b2901ce56207 started at 2022-11-24 23:26:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.965: INFO: Container busybox ready: false, restart count 0 Nov 24 23:26:49.965: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:26:42 +0000 UTC (0+7 container statuses recorded) Nov 24 23:26:49.965: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:49.965: INFO: netserver-1 started at 2022-11-24 23:26:34 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.965: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:49.965: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:49.965: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:49.965: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:49.965: INFO: netserver-1 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.965: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:49.965: INFO: hostexec-ip-172-20-43-101.us-east-2.compute.internal-ggfvk started at 2022-11-24 23:26:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:49.965: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:50.153: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:50.153: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:50.185: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 12458 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6618":"csi-mock-csi-mock-volumes-6618","ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:21:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:50.185: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:50.219: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:50.254: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:50.255: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:50.255: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:50.255: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:50.255: INFO: condition-test-zj4z4 started at 2022-11-24 23:26:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Container httpd ready: false, restart count 0 Nov 24 23:26:50.255: INFO: csi-mockplugin-0 started at 2022-11-24 23:25:04 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:50.255: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:26:50.255: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:26:50.255: INFO: Container mock ready: true, restart count 0 Nov 24 23:26:50.255: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:50.255: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:50.255: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Container coredns ready: true, restart count 0 Nov 24 23:26:50.255: INFO: netserver-2 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:50.255: INFO: netserver-2 started at 2022-11-24 23:26:34 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:50.255: INFO: pvc-volume-tester-b2p96 started at 2022-11-24 23:25:09 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Container volume-tester ready: true, restart count 0 Nov 24 23:26:50.255: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:25:04 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.255: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:26:50.425: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:50.425: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:50.456: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 14571 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6618":"ip-172-20-63-204.us-east-2.compute.internal","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 23:26:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:26:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:46 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:46 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:46 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:26:46 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6618^73f79354-6c4f-11ed-ba77-56d36fd5ac2f kubernetes.io/csi/csi-hostpath-ephemeral-6618^73f7e268-6c4f-11ed-ba77-56d36fd5ac2f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6618^73f79354-6c4f-11ed-ba77-56d36fd5ac2f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6618^73f7e268-6c4f-11ed-ba77-56d36fd5ac2f,DevicePath:,},},Config:nil,},} Nov 24 23:26:50.457: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:50.490: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:50.526: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.526: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:26:50.527: INFO: netserver-3 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:50.527: INFO: liveness-ea9ac8f0-3d79-40ff-9b86-3f2363c2d3bd started at 2022-11-24 23:23:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:50.527: INFO: netserver-3 started at 2022-11-24 23:26:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:50.527: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:50.527: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:50.527: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container coredns ready: true, restart count 0 Nov 24 23:26:50.527: INFO: test-webserver-8e667b28-a4ba-4e7e-9b6e-90264fc1e1b0 started at 2022-11-24 23:24:57 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container test-webserver ready: true, restart count 0 Nov 24 23:26:50.527: INFO: inline-volume-l7rcs started at 2022-11-24 23:25:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container volume-tester ready: false, restart count 0 Nov 24 23:26:50.527: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:50.527: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:26:39 +0000 UTC (0+7 container statuses recorded) Nov 24 23:26:50.527: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:50.527: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:50.527: INFO: inline-volume-tester-kf7kx started at 2022-11-24 23:26:43 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:50.527: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 23:26:50.713: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:50.713: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-5307" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 24 23:33:05.794: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc0036af030>: { s: "pod \"pod-640f2d10-3247-4271-8fd8-03da93bc8f80\" is not Running: timed out waiting for the condition", } pod "pod-640f2d10-3247-4271-8fd8-03da93bc8f80" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_13.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:28:05.441: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 24 23:28:05.585: INFO: Creating resource for dynamic PV Nov 24 23:28:05.585: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-9173-e2e-scfzdm7 �[1mSTEP�[0m: creating a claim Nov 24 23:28:05.615: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 24 23:33:05.794: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc0036af030>: { s: "pod \"pod-640f2d10-3247-4271-8fd8-03da93bc8f80\" is not Running: timed out waiting for the condition", } pod "pod-640f2d10-3247-4271-8fd8-03da93bc8f80" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000471800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000471800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000471800, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 24 23:33:05.795: INFO: Deleting pod "pod-640f2d10-3247-4271-8fd8-03da93bc8f80" in namespace "volume-expand-9173" Nov 24 23:33:05.828: INFO: Wait up to 5m0s for pod "pod-640f2d10-3247-4271-8fd8-03da93bc8f80" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 24 23:33:05.857: INFO: Deleting pod "pod-640f2d10-3247-4271-8fd8-03da93bc8f80" in namespace "volume-expand-9173" �[1mSTEP�[0m: Deleting pvc Nov 24 23:33:05.914: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comndvvc" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-9173". �[1mSTEP�[0m: Found 5 events. Nov 24 23:33:06.005: INFO: At 2022-11-24 23:28:05 +0000 UTC - event for ebs.csi.aws.comndvvc: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:33:06.005: INFO: At 2022-11-24 23:28:05 +0000 UTC - event for ebs.csi.aws.comndvvc: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:33:06.005: INFO: At 2022-11-24 23:28:05 +0000 UTC - event for ebs.csi.aws.comndvvc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-9173/ebs.csi.aws.comndvvc" Nov 24 23:33:06.005: INFO: At 2022-11-24 23:28:15 +0000 UTC - event for ebs.csi.aws.comndvvc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-9173-e2e-scfzdm7": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:33:06.005: INFO: At 2022-11-24 23:29:10 +0000 UTC - event for ebs.csi.aws.comndvvc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-9173-e2e-scfzdm7": rpc error: code = Internal desc = Could not create volume "pvc-2bc4a838-9aa8-4f31-b0bf-0c9da4e5a2a8": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 24 23:33:06.033: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:33:06.033: INFO: Nov 24 23:33:06.063: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:33:06.092: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 24685 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:32:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:32:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:32:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:32:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:33:06.092: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:33:06.123: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:33:06.163: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:33:06.164: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:33:06.164: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:33:06.164: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:33:06.164: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:33:06.164: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:33:06.164: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:06.164: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:33:06.164: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:33:06.164: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:33:06.164: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:06.164: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:33:06.164: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:33:06.164: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:33:06.164: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.164: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:33:06.321: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:33:06.321: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:33:06.350: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 23013 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-24 23:29:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:29:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:33:06.351: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:33:06.384: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-bdp96 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: affinity-clusterip-transition-h57dd started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-lbxg8 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-nrsr8 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:06.421: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:33:06.421: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:06.421: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-b8skk started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: pod-projected-configmaps-b9c822fd-5892-4041-940d-7c3d14342805 started at 2022-11-24 23:32:37 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-7nj8h started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-jkd2x started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: all-succeed-84gkf started at 2022-11-24 23:30:56 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container c ready: false, restart count 0 Nov 24 23:33:06.421: INFO: all-succeed-fb8tg started at 2022-11-24 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container c ready: false, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-d6lg5 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:33:06.421: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:33:06.421: INFO: pod-c39c6f03-f30b-48b1-95de-9abc2de560b5 started at 2022-11-24 23:32:11 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container test-container ready: false, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-vzj49 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-x7svl started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.421: INFO: pod-468ffb1a-c0c3-4446-af30-85e36c0935cf started at 2022-11-24 23:32:44 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container test-container ready: false, restart count 0 Nov 24 23:33:06.421: INFO: ss2-0 started at 2022-11-24 23:29:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container webserver ready: true, restart count 0 Nov 24 23:33:06.421: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-2fpvr started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.421: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.598: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:33:06.598: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:33:06.627: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 24030 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-8418":"ip-172-20-43-101.us-east-2.compute.internal","ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:31:00 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:31:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-8418^0c25b5d4-6c50-11ed-964c-6aeca698b218],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-8418^0c25b5d4-6c50-11ed-964c-6aeca698b218,DevicePath:,},},Config:nil,},} Nov 24 23:33:06.627: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:33:06.663: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:33:06.698: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:33:06.698: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:29:58 +0000 UTC (0+7 container statuses recorded) Nov 24 23:33:06.698: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-fbmch started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.698: INFO: my-hostname-basic-e7bd73af-a28d-41ad-9c02-a01481e6ab23-9wfgf started at 2022-11-24 23:30:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container my-hostname-basic-e7bd73af-a28d-41ad-9c02-a01481e6ab23 ready: false, restart count 0 Nov 24 23:33:06.698: INFO: ss-0 started at 2022-11-24 23:30:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container webserver ready: false, restart count 0 Nov 24 23:33:06.698: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:06.698: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:06.698: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-kpnff started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-ct29k started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.698: INFO: terminate-cmd-rpa3dea59ed-50f8-45f3-9058-9f2be1549f23 started at 2022-11-24 23:31:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container terminate-cmd-rpa ready: false, restart count 1 Nov 24 23:33:06.698: INFO: pfpod started at 2022-11-24 23:31:44 +0000 UTC (0+2 container statuses recorded) Nov 24 23:33:06.698: INFO: Container portforwardtester ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container readiness ready: false, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-r6lww started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-7gn8v started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.698: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:32:50 +0000 UTC (0+7 container statuses recorded) Nov 24 23:33:06.698: INFO: Container csi-attacher ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container csi-provisioner ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container csi-resizer ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container hostpath ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container liveness-probe ready: false, restart count 0 Nov 24 23:33:06.698: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6bw2w started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.698: INFO: all-succeed-pbfv2 started at 2022-11-24 23:30:51 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container c ready: false, restart count 0 Nov 24 23:33:06.698: INFO: all-succeed-crhdn started at 2022-11-24 23:30:55 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container c ready: false, restart count 0 Nov 24 23:33:06.698: INFO: ss2-1 started at 2022-11-24 23:31:49 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container webserver ready: false, restart count 0 Nov 24 23:33:06.698: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-nvkzw started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.698: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.699: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-42q46 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.699: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.699: INFO: httpd started at 2022-11-24 23:29:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.699: INFO: Container httpd ready: false, restart count 0 Nov 24 23:33:06.699: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-k4cjb started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.699: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.699: INFO: affinity-clusterip-transition-pqdk4 started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.699: INFO: Container affinity-clusterip-transition ready: false, restart count 0 Nov 24 23:33:06.699: INFO: pod-subpath-test-dynamicpv-h588 started at 2022-11-24 23:30:59 +0000 UTC (2+1 container statuses recorded) Nov 24 23:33:06.699: INFO: Init container init-volume-dynamicpv-h588 ready: false, restart count 0 Nov 24 23:33:06.699: INFO: Init container test-init-volume-dynamicpv-h588 ready: false, restart count 0 Nov 24 23:33:06.699: INFO: Container test-container-subpath-dynamicpv-h588 ready: false, restart count 0 Nov 24 23:33:06.699: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-2hsxc started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.699: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.877: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:33:06.877: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:33:06.907: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 21783 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:21:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{},"f:mounted_volume_expand":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:29:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:33:06.907: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:33:06.938: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:33:06.973: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:06.973: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:33:06.973: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:06.973: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:33:06.973: INFO: pod-subpath-test-preprovisionedpv-nm7j started at 2022-11-24 23:31:05 +0000 UTC (1+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Init container init-volume-preprovisionedpv-nm7j ready: false, restart count 0 Nov 24 23:33:06.973: INFO: Container test-container-subpath-preprovisionedpv-nm7j ready: false, restart count 0 Nov 24 23:33:06.973: INFO: csi-mockplugin-0 started at 2022-11-24 23:30:46 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:06.973: INFO: Container csi-provisioner ready: false, restart count 0 Nov 24 23:33:06.973: INFO: Container driver-registrar ready: false, restart count 0 Nov 24 23:33:06.973: INFO: Container mock ready: false, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6xwrp started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-69l56 started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-frgmv started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-5nzcw started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-dmtrc started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-xmxdf started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-qvzq5 started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-b876n started at 2022-11-24 23:30:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-lqfts started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:33:06.973: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:33:06.973: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container coredns ready: true, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-j7d5q started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-4ljg9 started at 2022-11-24 23:30:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:33:06.973: INFO: pod-5a5880cd-3940-42c2-9ae9-640bf68eebcd started at 2022-11-24 23:31:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container write-pod ready: false, restart count 0 Nov 24 23:33:06.973: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6hj9d started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:06.973: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:30:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container csi-attacher ready: false, restart count 0 Nov 24 23:33:06.973: INFO: affinity-clusterip-transition-67vl4 started at 2022-11-24 23:30:38 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:06.973: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Nov 24 23:33:07.155: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:33:07.155: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:33:07.184: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 24089 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9504":"csi-mock-csi-mock-volumes-9504","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-24 23:31:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:31:16 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9504^4],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:33:07.185: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:33:07.219: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:33:07.254: INFO: pvc-volume-tester-l444q started at 2022-11-24 23:31:09 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container volume-tester ready: false, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-5ljph started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-29zhg started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: false, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-tfmhr started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: csi-mockplugin-0 started at 2022-11-24 23:29:58 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:07.254: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:33:07.254: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:33:07.254: INFO: Container mock ready: true, restart count 0 Nov 24 23:33:07.254: INFO: pod-subpath-test-preprovisionedpv-cv7f started at 2022-11-24 23:31:51 +0000 UTC (1+2 container statuses recorded) Nov 24 23:33:07.254: INFO: Init container test-init-subpath-preprovisionedpv-cv7f ready: false, restart count 0 Nov 24 23:33:07.254: INFO: Container test-container-subpath-preprovisionedpv-cv7f ready: false, restart count 0 Nov 24 23:33:07.254: INFO: Container test-container-volume-preprovisionedpv-cv7f ready: false, restart count 0 Nov 24 23:33:07.254: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:29:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container csi-attacher ready: false, restart count 0 Nov 24 23:33:07.254: INFO: hostexec-ip-172-20-63-204.us-east-2.compute.internal-cfq8w started at 2022-11-24 23:31:34 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-tml7d started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-ntd7d started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-n994f started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-7ctz6 started at 2022-11-24 23:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:33:07.254: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:33:07.254: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:33:07.254: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:33:07.254: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container coredns ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-xt4tv started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-6sxsd started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7-g4dhj started at 2022-11-24 23:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Container cleanup40-d2a337a0-b989-4ba7-8aaa-29b1f57323b7 ready: true, restart count 0 Nov 24 23:33:07.254: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:33:07.254: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:33:07.254: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:33:07.430: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:33:07.430: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-9173" for this suite.
Find pod-640f2d10-3247-4271-8fd8-03da93bc8f80 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 24 23:28:32.455: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003812370>: { s: "pod \"pod-00cf2d94-6355-4fc5-a085-036d14a1bffd\" is not Running: timed out waiting for the condition", } pod "pod-00cf2d94-6355-4fc5-a085-036d14a1bffd" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_03.xml
{"msg":"PASSED [sig-cli] Kubectl client Guestbook application should create and stop a working application [Conformance]","total":-1,"completed":7,"skipped":33,"failed":0} [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:23:32.100: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 24 23:23:32.247: INFO: Creating resource for dynamic PV Nov 24 23:23:32.247: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-746-e2e-scg7hwb �[1mSTEP�[0m: creating a claim Nov 24 23:23:32.278: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 24 23:28:32.454: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003812370>: { s: "pod \"pod-00cf2d94-6355-4fc5-a085-036d14a1bffd\" is not Running: timed out waiting for the condition", } pod "pod-00cf2d94-6355-4fc5-a085-036d14a1bffd" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000470780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000470780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000470780, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 24 23:28:32.455: INFO: Deleting pod "pod-00cf2d94-6355-4fc5-a085-036d14a1bffd" in namespace "volume-expand-746" Nov 24 23:28:32.489: INFO: Wait up to 5m0s for pod "pod-00cf2d94-6355-4fc5-a085-036d14a1bffd" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 24 23:28:32.518: INFO: Deleting pod "pod-00cf2d94-6355-4fc5-a085-036d14a1bffd" in namespace "volume-expand-746" �[1mSTEP�[0m: Deleting pvc Nov 24 23:28:32.574: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comw9mh9" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-746". �[1mSTEP�[0m: Found 5 events. Nov 24 23:28:32.661: INFO: At 2022-11-24 23:23:32 +0000 UTC - event for ebs.csi.aws.comw9mh9: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:28:32.661: INFO: At 2022-11-24 23:23:32 +0000 UTC - event for ebs.csi.aws.comw9mh9: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:28:32.661: INFO: At 2022-11-24 23:23:32 +0000 UTC - event for ebs.csi.aws.comw9mh9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-746/ebs.csi.aws.comw9mh9" Nov 24 23:28:32.661: INFO: At 2022-11-24 23:23:42 +0000 UTC - event for ebs.csi.aws.comw9mh9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-746-e2e-scg7hwb": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:28:32.661: INFO: At 2022-11-24 23:23:53 +0000 UTC - event for ebs.csi.aws.comw9mh9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-746-e2e-scg7hwb": rpc error: code = Internal desc = Could not create volume "pvc-bdd969e2-410d-4c78-85f2-caf5a21d360d": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 24 23:28:32.688: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:28:32.689: INFO: Nov 24 23:28:32.718: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:28:32.747: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 18329 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:27:46 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:28:32.747: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:28:32.786: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:28:32.825: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:28:32.825: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:28:32.825: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:28:32.825: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:32.825: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:32.825: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:28:32.825: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:28:32.825: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:32.825: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:28:32.825: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:28:32.825: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:28:32.825: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:28:32.825: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:28:32.825: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:28:32.825: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:32.825: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:28:33.003: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:28:33.003: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:28:33.034: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 19885 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2687":"ip-172-20-41-205.us-east-2.compute.internal","csi-mock-csi-mock-volumes-3478":"csi-mock-csi-mock-volumes-3478","csi-mock-csi-mock-volumes-8173":"csi-mock-csi-mock-volumes-8173","ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 23:28:09 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:28:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:28:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:28:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:28:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:28:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-2687^abfdf65e-6c4f-11ed-a5aa-729e585e8843 kubernetes.io/csi/csi-mock-csi-mock-volumes-3478^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3478^4,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-2687^abfdf65e-6c4f-11ed-a5aa-729e585e8843,DevicePath:,},},Config:nil,},} Nov 24 23:28:33.034: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:28:33.068: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:28:33.107: INFO: pvc-volume-tester-n72dr started at 2022-11-24 23:27:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.107: INFO: Container volume-tester ready: true, restart count 0 Nov 24 23:28:33.107: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:28:33.107: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:28:33.107: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:28:33.107: INFO: hostexec-ip-172-20-41-205.us-east-2.compute.internal-2t26c started at 2022-11-24 23:28:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.107: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:28:33.108: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:33.108: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:33.108: INFO: csi-mockplugin-resizer-0 started at 2022-11-24 23:27:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.108: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:28:33.108: INFO: pvc-volume-tester-696zb started at 2022-11-24 23:28:08 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.108: INFO: Container volume-tester ready: true, restart count 0 Nov 24 23:28:33.108: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:28:10 +0000 UTC (0+7 container statuses recorded) Nov 24 23:28:33.108: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:33.108: INFO: concurrent-27822207-q7kzk started at 2022-11-24 23:27:00 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.108: INFO: Container c ready: true, restart count 0 Nov 24 23:28:33.108: INFO: pod-subpath-test-dynamicpv-w2t5 started at 2022-11-24 23:28:16 +0000 UTC (1+2 container statuses recorded) Nov 24 23:28:33.108: INFO: Init container test-init-subpath-dynamicpv-w2t5 ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container test-container-subpath-dynamicpv-w2t5 ready: false, restart count 0 Nov 24 23:28:33.108: INFO: Container test-container-volume-dynamicpv-w2t5 ready: false, restart count 0 Nov 24 23:28:33.108: INFO: deployment-shared-unset-55bfccbb6c-z4w28 started at 2022-11-24 23:28:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.108: INFO: Container nginx ready: false, restart count 0 Nov 24 23:28:33.108: INFO: csi-mockplugin-0 started at 2022-11-24 23:27:58 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:33.108: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container mock ready: true, restart count 0 Nov 24 23:28:33.108: INFO: csi-mockplugin-0 started at 2022-11-24 23:27:47 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:33.108: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:28:33.108: INFO: Container mock ready: true, restart count 0 Nov 24 23:28:33.108: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:27:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.108: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:28:33.108: INFO: pod-subpath-test-inlinevolume-g6pv started at 2022-11-24 23:28:29 +0000 UTC (1+2 container statuses recorded) Nov 24 23:28:33.108: INFO: Init container test-init-subpath-inlinevolume-g6pv ready: false, restart count 0 Nov 24 23:28:33.108: INFO: Container test-container-subpath-inlinevolume-g6pv ready: false, restart count 0 Nov 24 23:28:33.108: INFO: Container test-container-volume-inlinevolume-g6pv ready: false, restart count 0 Nov 24 23:28:33.379: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:28:33.379: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:28:33.409: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 19324 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:27:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:47 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:47 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:47 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:27:47 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:28:33.410: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:28:33.440: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:28:33.472: INFO: pod-subpath-test-secret-82f5 started at 2022-11-24 23:28:12 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Container test-container-subpath-secret-82f5 ready: true, restart count 0 Nov 24 23:28:33.472: INFO: pod-c095b0b1-9568-4b38-8b57-11ee5705b01e started at 2022-11-24 23:28:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Container write-pod ready: true, restart count 0 Nov 24 23:28:33.472: INFO: hostexec-ip-172-20-43-101.us-east-2.compute.internal-gfq8v started at 2022-11-24 23:28:23 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:28:33.472: INFO: deployment-shared-unset-55bfccbb6c-s4pgm started at 2022-11-24 23:28:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Container nginx ready: false, restart count 0 Nov 24 23:28:33.472: INFO: httpd-deployment-8584777d8-b57tg started at 2022-11-24 23:28:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Container httpd ready: true, restart count 0 Nov 24 23:28:33.472: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:28:33.472: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:28:33.472: INFO: httpd-deployment-948b4c64c-fmqh7 started at 2022-11-24 23:28:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.472: INFO: Container httpd ready: true, restart count 0 Nov 24 23:28:33.472: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:33.472: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:28:33.472: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:33.472: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:33.613: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:28:33.613: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:28:33.642: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 17637 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:21:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:28:33.642: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:28:33.673: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:28:33.711: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-zt7pz started at 2022-11-24 23:28:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:28:33.711: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:28:33.711: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:28:33.711: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Container coredns ready: true, restart count 0 Nov 24 23:28:33.711: INFO: local-client started at 2022-11-24 23:28:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Container local-client ready: true, restart count 0 Nov 24 23:28:33.711: INFO: httpd-deployment-948b4c64c-vsc5f started at 2022-11-24 23:28:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Container httpd ready: true, restart count 0 Nov 24 23:28:33.711: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:33.711: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:28:33.711: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:33.711: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:33.711: INFO: deployment-shared-unset-55bfccbb6c-65zx7 started at 2022-11-24 23:28:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Container nginx ready: false, restart count 0 Nov 24 23:28:33.711: INFO: test-cleanup-controller-wcgvw started at 2022-11-24 23:28:31 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:33.711: INFO: Container httpd ready: false, restart count 0 Nov 24 23:28:33.876: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:28:33.876: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:28:33.906: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 19930 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-4488":"ip-172-20-63-204.us-east-2.compute.internal","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:27:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:27:26 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:28:33.906: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:28:33.969: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:28:34.014: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:28:34.014: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:28:34.015: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:28:30 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:34.015: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:28:34.015: INFO: csi-mockplugin-0 started at 2022-11-24 23:28:30 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:34.015: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container mock ready: true, restart count 0 Nov 24 23:28:34.015: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:34.015: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:28:34.015: INFO: httpd-deployment-948b4c64c-75qf5 started at 2022-11-24 23:28:28 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:34.015: INFO: Container httpd ready: false, restart count 0 Nov 24 23:28:34.015: INFO: replace-27822208-jcrns started at 2022-11-24 23:28:00 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:34.015: INFO: Container c ready: true, restart count 0 Nov 24 23:28:34.015: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:28:26 +0000 UTC (0+7 container statuses recorded) Nov 24 23:28:34.015: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:34.015: INFO: dns-test-8339d1a8-5897-47b4-9b8e-090eb30fa0aa started at 2022-11-24 23:28:02 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:34.015: INFO: Container jessie-querier ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container querier ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container webserver ready: true, restart count 0 Nov 24 23:28:34.015: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:28:34.015: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:28:34.015: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:28:34.015: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:34.015: INFO: Container coredns ready: true, restart count 0 Nov 24 23:28:34.015: INFO: test-webserver-8e667b28-a4ba-4e7e-9b6e-90264fc1e1b0 started at 2022-11-24 23:24:57 +0000 UTC (0+1 container statuses recorded) Nov 24 23:28:34.015: INFO: Container test-webserver ready: true, restart count 0 Nov 24 23:28:34.258: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:28:34.258: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-746" for this suite.
Find pod-00cf2d94-6355-4fc5-a085-036d14a1bffd mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:26:31.980: Unexpected error: <*errors.errorString | 0xc003c62890>: { s: "pod \"pod-2c8a99f2-f7c6-45e0-acb1-67e2c835effc\" is not Running: timed out waiting for the condition", } pod "pod-2c8a99f2-f7c6-45e0-acb1-67e2c835effc" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_21.xml
{"msg":"PASSED [sig-network] Services should have session affinity timeout work for NodePort service [LinuxOnly] [Conformance]","total":-1,"completed":2,"skipped":39,"failed":0} [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:21:31.624: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:21:31.767: INFO: Creating resource for dynamic PV Nov 24 23:21:31.767: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-7663-e2e-scwd8b4 �[1mSTEP�[0m: creating a claim Nov 24 23:21:31.798: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-7663 with fsgroup 1000 Nov 24 23:26:31.980: FAIL: Unexpected error: <*errors.errorString | 0xc003c62890>: { s: "pod \"pod-2c8a99f2-f7c6-45e0-acb1-67e2c835effc\" is not Running: timed out waiting for the condition", } pod "pod-2c8a99f2-f7c6-45e0-acb1-67e2c835effc" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0036b18c0, 0xc0017290d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000615c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000615c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000615c80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 24 23:26:32.009: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comf5bwp" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-7663". �[1mSTEP�[0m: Found 4 events. Nov 24 23:26:32.098: INFO: At 2022-11-24 23:21:31 +0000 UTC - event for ebs.csi.aws.comf5bwp: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:26:32.098: INFO: At 2022-11-24 23:21:31 +0000 UTC - event for ebs.csi.aws.comf5bwp: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:26:32.098: INFO: At 2022-11-24 23:21:31 +0000 UTC - event for ebs.csi.aws.comf5bwp: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-7663/ebs.csi.aws.comf5bwp" Nov 24 23:26:32.098: INFO: At 2022-11-24 23:21:41 +0000 UTC - event for ebs.csi.aws.comf5bwp: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-7663-e2e-scwd8b4": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:26:32.126: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:26:32.126: INFO: pod-2c8a99f2-f7c6-45e0-acb1-67e2c835effc Pending [] Nov 24 23:26:32.126: INFO: Nov 24 23:26:32.155: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:32.184: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 7955 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:32.184: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:32.218: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:32.293: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:26:32.293: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:26:32.293: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:26:32.293: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:26:32.293: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:26:32.293: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:26:32.293: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:26:32.293: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:26:32.293: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:26:32.293: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:32.293: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:32.293: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.293: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:26:32.293: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:26:32.293: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:32.293: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:32.469: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:26:32.469: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:32.498: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 12801 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:22:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:32.499: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:32.530: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:32.561: INFO: pod-projected-configmaps-1cbd4a7c-7ed0-4a48-8b65-90d6489d8c89 started at 2022-11-24 23:25:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.561: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:32.561: INFO: affinity-clusterip-2wx4k started at 2022-11-24 23:26:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.561: INFO: Container affinity-clusterip ready: true, restart count 0 Nov 24 23:26:32.562: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:32.562: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:32.562: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:32.562: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:32.562: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:32.562: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:32.562: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:32.562: INFO: concurrent-27822206-4wvvm started at 2022-11-24 23:26:00 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.562: INFO: Container c ready: true, restart count 0 Nov 24 23:26:32.562: INFO: netserver-0 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.562: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:32.562: INFO: netserver-0 started at 2022-11-24 23:26:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.562: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:32.746: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:26:32.746: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:32.775: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 13378 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:26:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:32.775: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:32.807: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:32.839: INFO: busybox-c6872ff6-28c5-46f4-bd84-b2901ce56207 started at 2022-11-24 23:26:01 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Container busybox ready: false, restart count 0 Nov 24 23:26:32.839: INFO: execpod-affinityx8sgr started at 2022-11-24 23:26:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:32.839: INFO: netserver-1 started at 2022-11-24 23:26:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:32.839: INFO: pod-prestop-hook-2ea078d2-3db2-4575-9f8b-a601eb261674 started at 2022-11-24 23:26:11 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Container nginx ready: true, restart count 0 Nov 24 23:26:32.839: INFO: affinity-clusterip-trp52 started at 2022-11-24 23:26:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Container affinity-clusterip ready: true, restart count 0 Nov 24 23:26:32.839: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:32.839: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:32.839: INFO: netserver-1 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:32.839: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:32.839: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:32.839: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:32.839: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:32.839: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:32.985: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:26:32.985: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:33.014: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 12458 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6618":"csi-mock-csi-mock-volumes-6618","ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:21:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:24:09 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:33.014: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:33.045: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:33.078: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-kcv4b started at 2022-11-24 23:26:24 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:33.078: INFO: pvc-volume-tester-b2p96 started at 2022-11-24 23:25:09 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Container volume-tester ready: true, restart count 0 Nov 24 23:26:33.078: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:25:04 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:26:33.078: INFO: csi-mockplugin-0 started at 2022-11-24 23:25:04 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:33.078: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:26:33.078: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:26:33.078: INFO: Container mock ready: true, restart count 0 Nov 24 23:26:33.078: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:33.078: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:33.078: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:33.078: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:33.078: INFO: netserver-2 started at 2022-11-24 23:26:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:33.078: INFO: netserver-2 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:33.078: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:33.078: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:33.078: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.078: INFO: Container coredns ready: true, restart count 0 Nov 24 23:26:33.274: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:26:33.274: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:33.302: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 13576 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:25:06 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:26:33.303: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:33.333: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:33.368: INFO: netserver-3 started at 2022-11-24 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:33.368: INFO: liveness-ea9ac8f0-3d79-40ff-9b86-3f2363c2d3bd started at 2022-11-24 23:23:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:26:33.368: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:26:33.368: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:26:33.368: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:26:33.368: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:26:33.368: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container coredns ready: true, restart count 0 Nov 24 23:26:33.368: INFO: test-webserver-8e667b28-a4ba-4e7e-9b6e-90264fc1e1b0 started at 2022-11-24 23:24:57 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container test-webserver ready: true, restart count 0 Nov 24 23:26:33.368: INFO: inline-volume-l7rcs started at 2022-11-24 23:25:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container volume-tester ready: false, restart count 0 Nov 24 23:26:33.368: INFO: agnhost-primary-r65m7 started at 2022-11-24 23:26:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container agnhost-primary ready: true, restart count 0 Nov 24 23:26:33.368: INFO: affinity-clusterip-blwv4 started at 2022-11-24 23:26:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container affinity-clusterip ready: true, restart count 0 Nov 24 23:26:33.368: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:26:33.368: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:26:33.368: INFO: netserver-3 started at 2022-11-24 23:26:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container webserver ready: false, restart count 0 Nov 24 23:26:33.368: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:26:33.368: INFO: client-containers-d18f3b57-96af-45e6-b64b-4a73a7b967d9 started at 2022-11-24 23:26:30 +0000 UTC (0+1 container statuses recorded) Nov 24 23:26:33.368: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 23:26:34.983: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:26:34.983: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-7663" for this suite.
Find pod-2c8a99f2-f7c6-45e0-acb1-67e2c835effc mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:42:37.308: Unexpected error: <*errors.errorString | 0xc004476370>: { s: "pod \"pod-996cae3c-ce39-4f9a-bba6-7d14d866e271\" is not Running: timed out waiting for the condition", } pod "pod-996cae3c-ce39-4f9a-bba6-7d14d866e271" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_23.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:37:36.903: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:37:37.054: INFO: Creating resource for dynamic PV Nov 24 23:37:37.054: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-9234-e2e-schr2c6 �[1mSTEP�[0m: creating a claim Nov 24 23:37:37.086: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-9234 with fsgroup 1000 Nov 24 23:42:37.308: FAIL: Unexpected error: <*errors.errorString | 0xc004476370>: { s: "pod \"pod-996cae3c-ce39-4f9a-bba6-7d14d866e271\" is not Running: timed out waiting for the condition", } pod "pod-996cae3c-ce39-4f9a-bba6-7d14d866e271" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc003245760, 0xc002b5f0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00020ad80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00020ad80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00020ad80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 24 23:42:37.338: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comxhkfw" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-9234". �[1mSTEP�[0m: Found 5 events. Nov 24 23:42:37.428: INFO: At 2022-11-24 23:37:37 +0000 UTC - event for ebs.csi.aws.comxhkfw: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:42:37.428: INFO: At 2022-11-24 23:37:37 +0000 UTC - event for ebs.csi.aws.comxhkfw: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:42:37.428: INFO: At 2022-11-24 23:37:37 +0000 UTC - event for ebs.csi.aws.comxhkfw: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-9234/ebs.csi.aws.comxhkfw" Nov 24 23:42:37.428: INFO: At 2022-11-24 23:37:47 +0000 UTC - event for ebs.csi.aws.comxhkfw: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-9234-e2e-schr2c6": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:42:37.428: INFO: At 2022-11-24 23:38:42 +0000 UTC - event for ebs.csi.aws.comxhkfw: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-9234-e2e-schr2c6": rpc error: code = Internal desc = Could not create volume "pvc-104d05cc-a416-4717-9019-06d40051bbfb": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 24 23:42:37.457: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:42:37.457: INFO: pod-996cae3c-ce39-4f9a-bba6-7d14d866e271 Pending [] Nov 24 23:42:37.457: INFO: Nov 24 23:42:37.486: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:42:37.515: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 29255 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:37:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:37:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:37:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:37:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:42:37.515: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:42:37.548: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:42:37.588: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:42:37.588: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:42:37.588: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:42:37.588: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:37.588: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:42:37.588: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:42:37.588: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:42:37.588: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:42:37.588: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:42:37.588: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:42:37.588: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:42:37.588: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:42:37.588: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:42:37.588: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:42:37.588: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.588: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:42:37.766: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:42:37.766: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:42:37.794: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 40676 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9433":"csi-mock-csi-mock-volumes-9433","ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:42:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:42:37.795: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:42:37.826: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:42:37.859: INFO: hostexec-ip-172-20-41-205.us-east-2.compute.internal-2zqtk started at 2022-11-24 23:42:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.859: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:42:37.859: INFO: pod-subpath-test-preprovisionedpv-7rtb started at 2022-11-24 23:42:34 +0000 UTC (2+2 container statuses recorded) Nov 24 23:42:37.859: INFO: Init container init-volume-preprovisionedpv-7rtb ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Init container test-init-subpath-preprovisionedpv-7rtb ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Container test-container-subpath-preprovisionedpv-7rtb ready: false, restart count 0 Nov 24 23:42:37.859: INFO: Container test-container-volume-preprovisionedpv-7rtb ready: false, restart count 0 Nov 24 23:42:37.859: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:37.859: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:42:37.859: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:42:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.859: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:42:37.859: INFO: test-rolling-update-with-lb-5f85c5c5f4-rbxc5 started at 2022-11-24 23:41:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.859: INFO: Container agnhost ready: true, restart count 0 Nov 24 23:42:37.859: INFO: netserver-0 started at 2022-11-24 23:42:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:37.859: INFO: Container webserver ready: false, restart count 0 Nov 24 23:42:37.859: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:42:37.859: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:42:37.859: INFO: csi-mockplugin-0 started at 2022-11-24 23:42:13 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:37.859: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:42:37.859: INFO: Container mock ready: true, restart count 0 Nov 24 23:42:38.607: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:42:38.607: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:42:38.636: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 40691 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:40:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:42:38.636: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:42:38.668: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:42:38.699: INFO: pod-configmaps-3572b296-60c9-484d-97b0-86dd4d8ad9f8 started at 2022-11-24 23:41:33 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:38.699: INFO: Container createcm-volume-test ready: true, restart count 0 Nov 24 23:42:38.699: INFO: Container delcm-volume-test ready: true, restart count 0 Nov 24 23:42:38.699: INFO: Container updcm-volume-test ready: true, restart count 0 Nov 24 23:42:38.699: INFO: startup-ba348fb5-30f5-4be9-9e0a-6f1534f2e010 started at 2022-11-24 23:42:03 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.699: INFO: Container busybox ready: true, restart count 0 Nov 24 23:42:38.699: INFO: test-rolling-update-with-lb-5f85c5c5f4-nktvs started at 2022-11-24 23:41:43 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.699: INFO: Container agnhost ready: true, restart count 0 Nov 24 23:42:38.699: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:42:38.699: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:42:38.699: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:42:38.699: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:38.699: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:42:38.699: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:42:38.699: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:42:38.699: INFO: netserver-1 started at 2022-11-24 23:42:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.699: INFO: Container webserver ready: false, restart count 0 Nov 24 23:42:38.859: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:42:38.859: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:42:38.888: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 32822 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-24 23:38:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:42:38.888: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:42:38.919: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:42:38.951: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:42:38.951: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:42:38.951: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:42:38.951: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.951: INFO: Container coredns ready: true, restart count 0 Nov 24 23:42:38.951: INFO: successful-jobs-history-limit-27822222-kxzp2 started at 2022-11-24 23:42:00 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.951: INFO: Container c ready: false, restart count 0 Nov 24 23:42:38.951: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:38.951: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:42:38.951: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:42:38.951: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:42:38.951: INFO: netserver-2 started at 2022-11-24 23:42:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.951: INFO: Container webserver ready: false, restart count 0 Nov 24 23:42:38.951: INFO: rs-875qk started at 2022-11-24 23:40:37 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.951: INFO: Container donothing ready: false, restart count 0 Nov 24 23:42:38.951: INFO: test-rolling-update-with-lb-5f85c5c5f4-lx49h started at 2022-11-24 23:41:42 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:38.951: INFO: Container agnhost ready: true, restart count 0 Nov 24 23:42:39.116: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:42:39.117: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:42:39.145: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 40666 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:42:39.146: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:42:39.177: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:42:39.211: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:42:39.211: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:42:39.211: INFO: startup-b7ae2755-6499-4fef-866f-bdb70965f0aa started at 2022-11-24 23:42:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container busybox ready: false, restart count 0 Nov 24 23:42:39.211: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:42:39.211: INFO: netserver-3 started at 2022-11-24 23:42:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container webserver ready: false, restart count 0 Nov 24 23:42:39.211: INFO: busybox-7efc388b-68ac-45e6-b06c-f7d99a716d89 started at 2022-11-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container busybox ready: true, restart count 0 Nov 24 23:42:39.211: INFO: hostexec-ip-172-20-63-204.us-east-2.compute.internal-pl99n started at 2022-11-24 23:42:27 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:42:39.211: INFO: pod-subpath-test-preprovisionedpv-srlf started at 2022-11-24 23:42:34 +0000 UTC (2+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Init container init-volume-preprovisionedpv-srlf ready: true, restart count 0 Nov 24 23:42:39.211: INFO: Init container test-init-volume-preprovisionedpv-srlf ready: true, restart count 0 Nov 24 23:42:39.211: INFO: Container test-container-subpath-preprovisionedpv-srlf ready: false, restart count 0 Nov 24 23:42:39.211: INFO: pod-projected-configmaps-48b78809-9624-45fe-a63e-f8bff5779b7f started at 2022-11-24 23:42:37 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 23:42:39.211: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:42:39.211: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:42:39.211: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:42:39.211: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:42:39.211: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:42:39.211: INFO: Container coredns ready: true, restart count 0 Nov 24 23:42:42.201: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:42:42.201: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-9234" for this suite.
Find pod-996cae3c-ce39-4f9a-bba6-7d14d866e271 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:44:42.532: Unexpected error: <*errors.errorString | 0xc0057c50b0>: { s: "pod \"pod-9427b9d3-abe8-4433-9c34-83284f7951b0\" is not Running: timed out waiting for the condition", } pod "pod-9427b9d3-abe8-4433-9c34-83284f7951b0" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_19.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:39:42.174: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:39:42.319: INFO: Creating resource for dynamic PV Nov 24 23:39:42.319: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-486-e2e-scg25f9 �[1mSTEP�[0m: creating a claim Nov 24 23:39:42.348: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-486 with fsgroup 1000 Nov 24 23:44:42.532: FAIL: Unexpected error: <*errors.errorString | 0xc0057c50b0>: { s: "pod \"pod-9427b9d3-abe8-4433-9c34-83284f7951b0\" is not Running: timed out waiting for the condition", } pod "pod-9427b9d3-abe8-4433-9c34-83284f7951b0" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc00443f600, 0xc0068b10d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000b7bc80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000b7bc80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000b7bc80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 24 23:44:42.561: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comqnvlc" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-486". �[1mSTEP�[0m: Found 4 events. Nov 24 23:44:42.663: INFO: At 2022-11-24 23:39:42 +0000 UTC - event for ebs.csi.aws.comqnvlc: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:44:42.663: INFO: At 2022-11-24 23:39:42 +0000 UTC - event for ebs.csi.aws.comqnvlc: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:44:42.663: INFO: At 2022-11-24 23:39:42 +0000 UTC - event for ebs.csi.aws.comqnvlc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-486/ebs.csi.aws.comqnvlc" Nov 24 23:44:42.663: INFO: At 2022-11-24 23:39:52 +0000 UTC - event for ebs.csi.aws.comqnvlc: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-486-e2e-scg25f9": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:44:42.691: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:44:42.691: INFO: pod-9427b9d3-abe8-4433-9c34-83284f7951b0 Pending [] Nov 24 23:44:42.691: INFO: Nov 24 23:44:42.722: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:44:42.750: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 40981 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:44:42.751: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:44:42.783: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:44:42.845: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:44:42.845: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:44:42.845: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:44:42.845: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:44:42.845: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:44:42.845: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:44:42.845: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:44:42.845: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:44:42.845: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:44:42.845: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:44:42.845: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:44:42.845: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:42.845: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:44:42.845: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:44:42.845: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:44:42.845: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:44:43.018: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:44:43.018: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:44:43.048: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 41066 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:42:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:44:43.048: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:44:43.080: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:44:43.113: INFO: netserver-0 started at 2022-11-24 23:44:21 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.113: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.113: INFO: netserver-0 started at 2022-11-24 23:44:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.113: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.113: INFO: service-proxy-toggled-x5n4t started at 2022-11-24 23:44:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.113: INFO: Container service-proxy-toggled ready: true, restart count 0 Nov 24 23:44:43.113: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:43.113: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:44:43.113: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:44:43.113: INFO: affinity-clusterip-timeout-jnvz8 started at 2022-11-24 23:44:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.113: INFO: Container affinity-clusterip-timeout ready: true, restart count 0 Nov 24 23:44:43.113: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:44:43.113: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:44:43.113: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:44:43.113: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:44:43.273: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:44:43.273: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:44:43.301: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 44021 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:44:43.302: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:44:43.332: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:44:43.367: INFO: execpod-affinitylvz77 started at <nil> (0+0 container statuses recorded) Nov 24 23:44:43.367: INFO: pod-terminate-status-0-0 started at 2022-11-24 23:44:41 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Init container fail ready: false, restart count 0 Nov 24 23:44:43.367: INFO: Container blocked ready: false, restart count 0 Nov 24 23:44:43.367: INFO: test-ss-0 started at 2022-11-24 23:44:34 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.367: INFO: test-container-pod started at 2022-11-24 23:44:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.367: INFO: pod-terminate-status-2-0 started at 2022-11-24 23:44:41 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Init container fail ready: false, restart count 0 Nov 24 23:44:43.367: INFO: Container blocked ready: false, restart count 0 Nov 24 23:44:43.367: INFO: affinity-clusterip-timeout-6fv6t started at 2022-11-24 23:44:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container affinity-clusterip-timeout ready: true, restart count 0 Nov 24 23:44:43.367: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:44:43.367: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:44:43.367: INFO: netserver-1 started at 2022-11-24 23:44:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.367: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:44:43.367: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:44:43.367: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:44:43.367: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:44:43.367: INFO: netserver-1 started at 2022-11-24 23:44:21 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.367: INFO: service-proxy-toggled-xzkpc started at 2022-11-24 23:44:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container service-proxy-toggled ready: true, restart count 0 Nov 24 23:44:43.367: INFO: service-proxy-disabled-sbpgw started at 2022-11-24 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.367: INFO: Container service-proxy-disabled ready: true, restart count 0 Nov 24 23:44:43.620: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:44:43.620: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:44:43.649: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 42470 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-24 23:38:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:43:40 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:43:40 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:43:40 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:43:40 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:44:43.650: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:44:43.680: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:44:43.713: INFO: netserver-2 started at 2022-11-24 23:44:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.713: INFO: verify-service-up-host-exec-pod started at 2022-11-24 23:44:41 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:44:43.713: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:44:43.713: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:44:43.713: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:44:43.713: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:44:43.713: INFO: service-proxy-disabled-574w6 started at 2022-11-24 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container service-proxy-disabled ready: true, restart count 0 Nov 24 23:44:43.713: INFO: affinity-clusterip-timeout-l9x6z started at 2022-11-24 23:44:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container affinity-clusterip-timeout ready: true, restart count 0 Nov 24 23:44:43.713: INFO: netserver-2 started at 2022-11-24 23:44:21 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:43.713: INFO: service-proxy-toggled-v99d7 started at 2022-11-24 23:44:35 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container service-proxy-toggled ready: true, restart count 0 Nov 24 23:44:43.713: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:44:43.713: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:44:43.713: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container coredns ready: true, restart count 0 Nov 24 23:44:43.713: INFO: host-test-container-pod started at 2022-11-24 23:44:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:43.713: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:44:43.890: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:44:43.890: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:44:43.940: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 44025 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7355":"csi-mock-csi-mock-volumes-7355","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:44:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:17 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:17 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:17 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:44:17 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:44:43.941: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:44:43.997: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:44:44.063: INFO: busybox-7efc388b-68ac-45e6-b06c-f7d99a716d89 started at 2022-11-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.063: INFO: Container busybox ready: true, restart count 0 Nov 24 23:44:44.063: INFO: netserver-3 started at 2022-11-24 23:44:22 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.063: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:44.063: INFO: service-proxy-disabled-nwpqr started at 2022-11-24 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.063: INFO: Container service-proxy-disabled ready: true, restart count 0 Nov 24 23:44:44.063: INFO: inline-volume-tester-hdzrt started at <nil> (0+0 container statuses recorded) Nov 24 23:44:44.063: INFO: csi-mockplugin-resizer-0 started at 2022-11-24 23:44:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.063: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:44:44.063: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:44:44.063: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:44:44.063: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:44:44.063: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:44:44.064: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.064: INFO: Container coredns ready: true, restart count 0 Nov 24 23:44:44.064: INFO: netserver-3 started at 2022-11-24 23:44:14 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.064: INFO: Container webserver ready: true, restart count 0 Nov 24 23:44:44.064: INFO: csi-mockplugin-0 started at 2022-11-24 23:44:33 +0000 UTC (0+3 container statuses recorded) Nov 24 23:44:44.064: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:44:44.064: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:44:44.064: INFO: Container mock ready: true, restart count 0 Nov 24 23:44:44.064: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:44:44.064: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:44:44.064: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:44:44.064: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.064: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:44:44.064: INFO: pod-handle-http-request started at 2022-11-24 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 24 23:44:44.064: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:44:44.064: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:44:43 +0000 UTC (0+7 container statuses recorded) Nov 24 23:44:44.064: INFO: Container csi-attacher ready: false, restart count 0 Nov 24 23:44:44.064: INFO: Container csi-provisioner ready: false, restart count 0 Nov 24 23:44:44.064: INFO: Container csi-resizer ready: false, restart count 0 Nov 24 23:44:44.064: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 24 23:44:44.064: INFO: Container hostpath ready: false, restart count 0 Nov 24 23:44:44.064: INFO: Container liveness-probe ready: false, restart count 0 Nov 24 23:44:44.064: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 24 23:44:45.559: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:44:45.559: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-486" for this suite.
Find pod-9427b9d3-abe8-4433-9c34-83284f7951b0 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:43:15.662: Unexpected error: <*errors.errorString | 0xc0023f1970>: { s: "pod \"pod-f929092a-b7b9-49bf-8bca-5a54b74a06cb\" is not Running: timed out waiting for the condition", } pod "pod-f929092a-b7b9-49bf-8bca-5a54b74a06cb" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_11.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:38:15.297: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:38:15.446: INFO: Creating resource for dynamic PV Nov 24 23:38:15.446: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-7846-e2e-sc6nghk �[1mSTEP�[0m: creating a claim Nov 24 23:38:15.478: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-7846 with fsgroup 1000 Nov 24 23:43:15.662: FAIL: Unexpected error: <*errors.errorString | 0xc0023f1970>: { s: "pod \"pod-f929092a-b7b9-49bf-8bca-5a54b74a06cb\" is not Running: timed out waiting for the condition", } pod "pod-f929092a-b7b9-49bf-8bca-5a54b74a06cb" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc00338a2c0, 0xc00328b0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000d6a480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000d6a480) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000d6a480, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 24 23:43:15.694: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comchg7g" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-7846". �[1mSTEP�[0m: Found 4 events. Nov 24 23:43:15.785: INFO: At 2022-11-24 23:38:15 +0000 UTC - event for ebs.csi.aws.comchg7g: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:43:15.785: INFO: At 2022-11-24 23:38:15 +0000 UTC - event for ebs.csi.aws.comchg7g: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:43:15.785: INFO: At 2022-11-24 23:38:15 +0000 UTC - event for ebs.csi.aws.comchg7g: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-7846/ebs.csi.aws.comchg7g" Nov 24 23:43:15.785: INFO: At 2022-11-24 23:38:25 +0000 UTC - event for ebs.csi.aws.comchg7g: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-7846-e2e-sc6nghk": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:43:15.818: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:43:15.818: INFO: pod-f929092a-b7b9-49bf-8bca-5a54b74a06cb Pending [] Nov 24 23:43:15.818: INFO: Nov 24 23:43:15.848: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:43:15.878: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 40981 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:43:15.879: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:43:15.912: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:43:15.946: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:43:15.946: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:43:15.946: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:43:15.946: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:43:15.946: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:43:15.946: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:43:15.946: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:43:15.946: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:43:15.946: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:43:15.946: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:15.946: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:43:15.946: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:43:15.946: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:15.946: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:43:15.946: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:43:15.946: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:43:16.099: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:43:16.099: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:43:16.130: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 41066 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:42:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:43:16.130: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:43:16.162: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:43:16.208: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:43:16.208: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:43:16.209: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:16.209: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:43:16.209: INFO: pod-client started at 2022-11-24 23:43:07 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.209: INFO: Container pod-client ready: true, restart count 0 Nov 24 23:43:16.209: INFO: hostexec-ip-172-20-41-205.us-east-2.compute.internal-th8rd started at 2022-11-24 23:43:15 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.209: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 23:43:16.209: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:43:16.209: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:43:16.209: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:43:16.390: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:43:16.390: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:43:16.419: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 40691 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:40:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:43:16.420: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:43:16.452: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:43:16.484: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:43:16.485: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:43:16.485: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:16.485: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:43:16.485: INFO: local-client started at 2022-11-24 23:43:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.485: INFO: Container local-client ready: false, restart count 0 Nov 24 23:43:16.485: INFO: pfpod started at 2022-11-24 23:43:14 +0000 UTC (0+2 container statuses recorded) Nov 24 23:43:16.485: INFO: Container portforwardtester ready: false, restart count 0 Nov 24 23:43:16.485: INFO: Container readiness ready: false, restart count 0 Nov 24 23:43:16.485: INFO: hostexec-ip-172-20-43-101.us-east-2.compute.internal-9c2kq started at 2022-11-24 23:42:53 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.485: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:43:16.485: INFO: busybox-privileged-true-dd359318-cd87-4e94-aa66-9716d5f6b9ac started at 2022-11-24 23:43:09 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.485: INFO: Container busybox-privileged-true-dd359318-cd87-4e94-aa66-9716d5f6b9ac ready: false, restart count 0 Nov 24 23:43:16.485: INFO: pod-subpath-test-projected-wqfk started at 2022-11-24 23:42:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.485: INFO: Container test-container-subpath-projected-wqfk ready: false, restart count 0 Nov 24 23:43:16.485: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:43:16.485: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:43:16.485: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:43:16.647: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:43:16.647: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:43:16.678: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 32822 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-24 23:38:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:38:30 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:43:16.678: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:43:16.710: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:43:16.743: INFO: downwardapi-volume-c1e64302-cb38-4aeb-a980-b016553718f0 started at 2022-11-24 23:43:15 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.743: INFO: Container client-container ready: false, restart count 0 Nov 24 23:43:16.743: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:43:16.743: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:43:16.743: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:43:16.743: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:16.743: INFO: Container coredns ready: true, restart count 0 Nov 24 23:43:16.743: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:43:16.743: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:43:16.743: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:16.743: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:43:16.743: INFO: pfpod started at 2022-11-24 23:43:01 +0000 UTC (0+2 container statuses recorded) Nov 24 23:43:16.743: INFO: Container portforwardtester ready: false, restart count 0 Nov 24 23:43:16.743: INFO: Container readiness ready: false, restart count 0 Nov 24 23:43:16.918: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:43:16.918: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:43:16.947: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 41586 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-7574":"ip-172-20-63-204.us-east-2.compute.internal","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 23:42:52 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:42:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:43:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:43:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:43:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:43:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-7574^b3cc402a-6c51-11ed-af42-6e991159e0cd kubernetes.io/csi/csi-hostpath-ephemeral-7574^bb5adb29-6c51-11ed-af42-6e991159e0cd],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-7574^b3cc402a-6c51-11ed-af42-6e991159e0cd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-7574^bb5adb29-6c51-11ed-af42-6e991159e0cd,DevicePath:,},},Config:nil,},} Nov 24 23:43:16.948: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:43:16.979: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:43:17.013: INFO: startup-b7ae2755-6499-4fef-866f-bdb70965f0aa started at 2022-11-24 23:42:26 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container busybox ready: true, restart count 0 Nov 24 23:43:17.013: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:43:17.013: INFO: hostexec-ip-172-20-63-204.us-east-2.compute.internal-4qvlj started at 2022-11-24 23:42:50 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:43:17.013: INFO: inline-volume-tester-c69g8 started at 2022-11-24 23:42:51 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 23:43:17.013: INFO: inline-volume-tester2-p28pm started at 2022-11-24 23:43:02 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 23:43:17.013: INFO: busybox-7efc388b-68ac-45e6-b06c-f7d99a716d89 started at 2022-11-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container busybox ready: true, restart count 0 Nov 24 23:43:17.013: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container coredns ready: true, restart count 0 Nov 24 23:43:17.013: INFO: local-client started at 2022-11-24 23:43:11 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container local-client ready: true, restart count 0 Nov 24 23:43:17.013: INFO: pod-server-1 started at 2022-11-24 23:43:12 +0000 UTC (0+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:43:17.013: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:43:17.013: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:43:17.013: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:42:44 +0000 UTC (0+7 container statuses recorded) Nov 24 23:43:17.013: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:43:17.013: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:43:17.013: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:43:17.013: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:43:17.220: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:43:17.220: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-7846" for this suite.
Find pod-f929092a-b7b9-49bf-8bca-5a54b74a06cb mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:45:52.407: Unexpected error: <*errors.errorString | 0xc004ba4fc0>: { s: "pod \"pod-76470b4b-eb1a-4bbb-aeca-03b2ec5de77c\" is not Running: timed out waiting for the condition", } pod "pod-76470b4b-eb1a-4bbb-aeca-03b2ec5de77c" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_04.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:40:52.062: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:40:52.204: INFO: Creating resource for dynamic PV Nov 24 23:40:52.204: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-6120-e2e-scxfhq8 �[1mSTEP�[0m: creating a claim Nov 24 23:40:52.234: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-6120 with fsgroup 1000 Nov 24 23:45:52.407: FAIL: Unexpected error: <*errors.errorString | 0xc004ba4fc0>: { s: "pod \"pod-76470b4b-eb1a-4bbb-aeca-03b2ec5de77c\" is not Running: timed out waiting for the condition", } pod "pod-76470b4b-eb1a-4bbb-aeca-03b2ec5de77c" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc003f23080, 0xc0020690d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000322d80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000322d80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000322d80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 24 23:45:52.435: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comtktkt" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-6120". �[1mSTEP�[0m: Found 4 events. Nov 24 23:45:52.526: INFO: At 2022-11-24 23:40:52 +0000 UTC - event for ebs.csi.aws.comtktkt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:45:52.526: INFO: At 2022-11-24 23:40:52 +0000 UTC - event for ebs.csi.aws.comtktkt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:45:52.526: INFO: At 2022-11-24 23:40:52 +0000 UTC - event for ebs.csi.aws.comtktkt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-6120/ebs.csi.aws.comtktkt" Nov 24 23:45:52.526: INFO: At 2022-11-24 23:41:02 +0000 UTC - event for ebs.csi.aws.comtktkt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-6120-e2e-scxfhq8": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:45:52.554: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:45:52.554: INFO: pod-76470b4b-eb1a-4bbb-aeca-03b2ec5de77c Pending [] Nov 24 23:45:52.554: INFO: Nov 24 23:45:52.584: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:52.613: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 40981 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:52.613: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:52.644: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:52.682: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:45:52.682: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:45:52.682: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:45:52.682: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:45:52.682: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:45:52.682: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:45:52.682: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:45:52.682: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:45:52.682: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:45:52.682: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:52.682: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:52.682: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.682: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:45:52.682: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:45:52.682: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:52.682: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:52.841: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:52.841: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:52.870: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 46236 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:45:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:52.870: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:52.907: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:52.938: INFO: netserver-0 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.938: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:52.938: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:52.938: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:52.938: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:52.938: INFO: pod-terminate-status-1-14 started at 2022-11-24 23:45:49 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:52.938: INFO: Init container fail ready: false, restart count 0 Nov 24 23:45:52.938: INFO: Container blocked ready: false, restart count 0 Nov 24 23:45:52.938: INFO: pod-terminate-status-0-13 started at 2022-11-24 23:45:51 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:52.938: INFO: Init container fail ready: false, restart count 0 Nov 24 23:45:52.938: INFO: Container blocked ready: false, restart count 0 Nov 24 23:45:52.938: INFO: hostexec-ip-172-20-41-205.us-east-2.compute.internal-qw8d4 started at 2022-11-24 23:45:48 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.938: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:45:52.938: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:52.938: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:52.938: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:52.938: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:52.938: INFO: netserver-0 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:52.938: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:55.867: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:55.867: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:55.896: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 44021 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:55.896: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:55.926: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:55.958: INFO: pod-0 started at 2022-11-24 23:45:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Container donothing ready: true, restart count 0 Nov 24 23:45:55.958: INFO: webserver-pod started at <nil> (0+0 container statuses recorded) Nov 24 23:45:55.958: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:55.958: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:55.958: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:55.958: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:55.958: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:55.958: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:55.958: INFO: pod1 started at 2022-11-24 23:45:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 23:45:55.958: INFO: netserver-1 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:55.958: INFO: netserver-1 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:55.958: INFO: pod-init-834fe06d-9b03-444c-89f6-94177997eade started at 2022-11-24 23:45:13 +0000 UTC (2+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Init container init1 ready: false, restart count 2 Nov 24 23:45:55.958: INFO: Init container init2 ready: false, restart count 0 Nov 24 23:45:55.958: INFO: Container run1 ready: false, restart count 0 Nov 24 23:45:55.958: INFO: test-container-pod started at 2022-11-24 23:45:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:55.958: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:57.045: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:57.045: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:57.073: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 46498 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-24 23:45:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:50 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:50 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:50 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:50 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:57.076: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:57.113: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:57.151: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:57.151: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:57.151: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:57.151: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:57.151: INFO: implicit-nonroot-uid started at 2022-11-24 23:45:56 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Nov 24 23:45:57.151: INFO: pod-subpath-test-preprovisionedpv-mtbd started at 2022-11-24 23:45:49 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Init container init-volume-preprovisionedpv-mtbd ready: true, restart count 0 Nov 24 23:45:57.151: INFO: Container test-container-subpath-preprovisionedpv-mtbd ready: true, restart count 0 Nov 24 23:45:57.151: INFO: netserver-2 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:57.151: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:57.151: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:57.151: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Container coredns ready: true, restart count 0 Nov 24 23:45:57.151: INFO: netserver-2 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:57.151: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-kc2j8 started at 2022-11-24 23:45:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.151: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:45:57.438: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:57.438: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:57.466: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 45900 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7355":"csi-mock-csi-mock-volumes-7355","csi-mock-csi-mock-volumes-9948":"csi-mock-csi-mock-volumes-9948","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 23:45:02 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:45:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9948^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-9948^4,DevicePath:,},},Config:nil,},} Nov 24 23:45:57.467: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:57.500: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:57.533: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:45:57.533: INFO: netserver-3 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:57.533: INFO: hostexec-ip-172-20-63-204.us-east-2.compute.internal-9hkt7 started at 2022-11-24 23:45:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:45:57.533: INFO: csi-mockplugin-resizer-0 started at 2022-11-24 23:44:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:57.533: INFO: csi-mockplugin-resizer-0 started at 2022-11-24 23:44:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:57.533: INFO: busybox-7efc388b-68ac-45e6-b06c-f7d99a716d89 started at 2022-11-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container busybox ready: true, restart count 0 Nov 24 23:45:57.533: INFO: netserver-3 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:57.533: INFO: pvc-volume-tester-n5hvp started at 2022-11-24 23:44:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container volume-tester ready: true, restart count 0 Nov 24 23:45:57.533: INFO: csi-mockplugin-0 started at 2022-11-24 23:44:33 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:57.533: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container mock ready: true, restart count 0 Nov 24 23:45:57.533: INFO: csi-mockplugin-0 started at 2022-11-24 23:44:46 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:57.533: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container mock ready: true, restart count 0 Nov 24 23:45:57.533: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:57.533: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:57.533: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container coredns ready: true, restart count 0 Nov 24 23:45:57.533: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:57.533: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:57.533: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:44:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:57.533: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:45:57.780: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:57.780: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-6120" for this suite.
Find pod-76470b4b-eb1a-4bbb-aeca-03b2ec5de77c mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sskips\sownership\schanges\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:45:46.175: Unexpected error: <*errors.errorString | 0xc003826fe0>: { s: "pod \"pod-4c223c5d-4892-4eae-9151-ef35f1d1af3f\" is not Running: timed out waiting for the condition", } pod "pod-4c223c5d-4892-4eae-9151-ef35f1d1af3f" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_05.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:40:45.817: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 24 23:40:45.964: INFO: Creating resource for dynamic PV Nov 24 23:40:45.964: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-9077-e2e-scpjngh �[1mSTEP�[0m: creating a claim Nov 24 23:40:45.994: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-9077 with fsgroup 1000 Nov 24 23:45:46.175: FAIL: Unexpected error: <*errors.errorString | 0xc003826fe0>: { s: "pod \"pod-4c223c5d-4892-4eae-9151-ef35f1d1af3f\" is not Running: timed out waiting for the condition", } pod "pod-4c223c5d-4892-4eae-9151-ef35f1d1af3f" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc00377d600, 0xc0006510d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000602780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000602780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000602780, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 24 23:45:46.205: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comhx8cm" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-9077". �[1mSTEP�[0m: Found 4 events. Nov 24 23:45:46.299: INFO: At 2022-11-24 23:40:46 +0000 UTC - event for ebs.csi.aws.comhx8cm: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:45:46.299: INFO: At 2022-11-24 23:40:46 +0000 UTC - event for ebs.csi.aws.comhx8cm: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:45:46.299: INFO: At 2022-11-24 23:40:46 +0000 UTC - event for ebs.csi.aws.comhx8cm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-9077/ebs.csi.aws.comhx8cm" Nov 24 23:45:46.299: INFO: At 2022-11-24 23:40:56 +0000 UTC - event for ebs.csi.aws.comhx8cm: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-9077-e2e-scpjngh": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:45:46.328: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:45:46.328: INFO: pod-4c223c5d-4892-4eae-9151-ef35f1d1af3f Pending [] Nov 24 23:45:46.328: INFO: Nov 24 23:45:46.359: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:46.388: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 40981 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:42:47 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:46.389: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:46.420: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:46.454: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:45:46.454: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:45:46.454: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.454: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:45:46.454: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:46.454: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:46.454: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.454: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:45:46.454: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:45:46.454: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:46.454: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:46.454: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.454: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:45:46.454: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.454: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:45:46.454: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.454: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:45:46.454: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:46.454: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:46.455: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:45:46.455: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.455: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:45:46.455: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.455: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:45:46.594: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:45:46.594: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:46.624: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 45974 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-5186":"csi-mock-csi-mock-volumes-5186","ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:45:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:46.624: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:46.656: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:46.696: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:46.696: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:46.696: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:46.696: INFO: csi-mockplugin-0 started at 2022-11-24 23:45:13 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:46.696: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:46.696: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:45:46.696: INFO: Container mock ready: true, restart count 0 Nov 24 23:45:46.696: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:45:13 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.696: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:45:46.696: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:46.696: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:46.696: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:46.696: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:46.696: INFO: netserver-0 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.696: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:46.696: INFO: netserver-0 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:46.696: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:46.907: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:45:46.907: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:46.937: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 44021 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:43:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:44:37 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:45:46.938: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:46.969: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:47.003: INFO: dns-test-b34ea56b-9c1a-4574-9fb7-918d04929fa4 started at 2022-11-24 23:45:39 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:47.003: INFO: Container jessie-querier ready: false, restart count 0 Nov 24 23:45:47.003: INFO: Container querier ready: false, restart count 0 Nov 24 23:45:47.003: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:47.003: INFO: pod-terminate-status-1-13 started at <nil> (0+0 container statuses recorded) Nov 24 23:45:47.003: INFO: pod-0 started at <nil> (0+0 container statuses recorded) Nov 24 23:45:47.003: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:47.003: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:47.003: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:47.003: INFO: ebs-csi-node-czqtv started at 2022-11-24 23:17:07 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:47.003: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:47.003: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:47.003: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:47.003: INFO: netserver-1 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:47.003: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:47.003: INFO: netserver-1 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:47.003: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:47.003: INFO: pod-init-834fe06d-9b03-444c-89f6-94177997eade started at 2022-11-24 23:45:13 +0000 UTC (2+1 container statuses recorded) Nov 24 23:45:47.003: INFO: Init container init1 ready: false, restart count 2 Nov 24 23:45:47.003: INFO: Init container init2 ready: false, restart count 0 Nov 24 23:45:47.003: INFO: Container run1 ready: false, restart count 0 Nov 24 23:45:47.003: INFO: pod-terminate-status-2-13 started at <nil> (0+0 container statuses recorded) Nov 24 23:45:47.003: INFO: pod-terminate-status-0-12 started at <nil> (0+0 container statuses recorded) Nov 24 23:45:47.639: INFO: Latency metrics for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:45:47.639: INFO: Logging node info for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:47.670: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-182.us-east-2.compute.internal f57fa410-594e-4385-926c-7029a71c4756 45862 0 2022-11-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-182.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-7684 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-182.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6470":"ip-172-20-43-182.us-east-2.compute.internal","ebs.csi.aws.com":"i-08402f9d9364f171d"} io.cilium.network.ipv4-cilium-host:172.20.36.221 io.cilium.network.ipv4-health-ip:172.20.46.88 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-24 23:29:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:45:05 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:45:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08402f9d9364f171d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:35 +0000 UTC,LastTransitionTime:2022-11-24 23:17:35 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:10 +0000 UTC,LastTransitionTime:2022-11-24 23:17:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.182,},NodeAddress{Type:ExternalIP,Address:3.144.249.174,},NodeAddress{Type:Hostname,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-182.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-144-249-174.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21b4df0b180f067f0a52e6a4003bf7,SystemUUID:ec21b4df-0b18-0f06-7f0a-52e6a4003bf7,BootID:1fefd8fc-7929-4012-a0a3-b413d4ac54be,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6470^051668b0-6c52-11ed-b3de-8a37c0f71dcb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6470^051668b0-6c52-11ed-b3de-8a37c0f71dcb,DevicePath:,},},Config:nil,},} Nov 24 23:45:47.670: INFO: Logging kubelet events for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:47.702: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:47.736: INFO: csi-hostpathplugin-0 started at 2022-11-24 23:45:01 +0000 UTC (0+7 container statuses recorded) Nov 24 23:45:47.736: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container hostpath ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:47.736: INFO: cilium-xdjz7 started at 2022-11-24 23:17:09 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:47.736: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:47.736: INFO: coredns-5fcc7b6498-nb22d started at 2022-11-24 23:17:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:47.736: INFO: Container coredns ready: true, restart count 0 Nov 24 23:45:47.736: INFO: netserver-2 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:47.736: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:47.736: INFO: netserver-2 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:47.736: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:47.736: INFO: hostexec-ip-172-20-43-182.us-east-2.compute.internal-kc2j8 started at 2022-11-24 23:45:32 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:47.736: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:45:47.736: INFO: ebs-csi-node-5p6kq started at 2022-11-24 23:17:09 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:47.736: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:47.736: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:47.950: INFO: Latency metrics for node ip-172-20-43-182.us-east-2.compute.internal Nov 24 23:45:47.950: INFO: Logging node info for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:47.983: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-63-204.us-east-2.compute.internal ed42168c-77af-4563-8d6e-38e92b664de4 45900 0 2022-11-24 23:17:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-63-204.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-63-204.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7355":"csi-mock-csi-mock-volumes-7355","csi-mock-csi-mock-volumes-9948":"csi-mock-csi-mock-volumes-9948","ebs.csi.aws.com":"i-03512e1700ec90e1a"} io.cilium.network.ipv4-cilium-host:172.20.52.221 io.cilium.network.ipv4-health-ip:172.20.32.25 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 23:45:02 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:45:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-03512e1700ec90e1a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:45:07 +0000 UTC,LastTransitionTime:2022-11-24 23:17:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.204,},NodeAddress{Type:ExternalIP,Address:3.15.27.135,},NodeAddress{Type:Hostname,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-63-204.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-15-27-135.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f7b7e293efb5271b985538ef87264,SystemUUID:ec2f7b7e-293e-fb52-71b9-85538ef87264,BootID:cd2570a0-12cc-42ea-b951-3f1b54295739,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9948^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-9948^4,DevicePath:,},},Config:nil,},} Nov 24 23:45:47.983: INFO: Logging kubelet events for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:48.015: INFO: Logging pods the kubelet thinks is on node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:48.077: INFO: csi-mockplugin-resizer-0 started at 2022-11-24 23:44:33 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.077: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:48.077: INFO: csi-mockplugin-resizer-0 started at 2022-11-24 23:44:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.077: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:45:48.077: INFO: busybox-7efc388b-68ac-45e6-b06c-f7d99a716d89 started at 2022-11-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.077: INFO: Container busybox ready: true, restart count 0 Nov 24 23:45:48.077: INFO: netserver-3 started at 2022-11-24 23:45:05 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.077: INFO: Container webserver ready: true, restart count 0 Nov 24 23:45:48.077: INFO: pvc-volume-tester-gm2nn started at 2022-11-24 23:44:58 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.077: INFO: Container volume-tester ready: false, restart count 0 Nov 24 23:45:48.077: INFO: pvc-volume-tester-n5hvp started at 2022-11-24 23:44:45 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.078: INFO: Container volume-tester ready: true, restart count 0 Nov 24 23:45:48.078: INFO: csi-mockplugin-0 started at 2022-11-24 23:44:33 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:48.078: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container mock ready: true, restart count 0 Nov 24 23:45:48.078: INFO: csi-mockplugin-0 started at 2022-11-24 23:44:46 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:48.078: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container mock ready: true, restart count 0 Nov 24 23:45:48.078: INFO: ebs-csi-node-g6dbm started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:45:48.078: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:45:48.078: INFO: coredns-5fcc7b6498-m9c98 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.078: INFO: Container coredns ready: true, restart count 0 Nov 24 23:45:48.078: INFO: cilium-x585z started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:45:48.078: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:45:48.078: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:45:48.078: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 23:44:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.078: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:45:48.078: INFO: coredns-autoscaler-6658b4bf85-4dld4 started at 2022-11-24 23:17:25 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.078: INFO: Container autoscaler ready: true, restart count 0 Nov 24 23:45:48.078: INFO: netserver-3 started at 2022-11-24 23:45:40 +0000 UTC (0+1 container statuses recorded) Nov 24 23:45:48.078: INFO: Container webserver ready: false, restart count 0 Nov 24 23:45:48.341: INFO: Latency metrics for node ip-172-20-63-204.us-east-2.compute.internal Nov 24 23:45:48.341: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-9077" for this suite.
Find pod-4c223c5d-4892-4eae-9151-ef35f1d1af3f mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\smount\soptions$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 24 23:27:12.983: Unexpected error: <*errors.errorString | 0xc00242b830>: { s: "pod \"pod-51dcf2d0-d659-427d-81d0-d42a6e074dad\" is not Running: timed out waiting for the condition", } pod "pod-51dcf2d0-d659-427d-81d0-d42a6e074dad" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 24 23:22:12.564: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should provision storage with mount options /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 24 23:22:12.713: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 24 23:22:12.713: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 24 23:22:12.713: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-6376-e2e-scfxkk9 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-6376 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-6376-e2e-scfxkk9,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-6376 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-6376-e2e-scfxkk9,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-6376 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-6376-e2e-scfxkk9,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-6376-e2e-scfxkk9 f1504615-2b3e-464c-bb9b-6c452bb92cec 6655 0 2022-11-24 23:22:12 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-11-24 23:22:12 +0000 UTC FieldsV1 {"f:allowVolumeExpansion":{},"f:mountOptions":{},"f:parameters":{".":{},"f:encrypted":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:*Delete,MountOptions:[dirsync],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-m7cnr pvc- provisioning-6376 f80e7a50-6b41-4373-953f-209675f2f086 6656 0 2022-11-24 23:22:12 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-11-24 23:22:12 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-6376-e2e-scfxkk9,VolumeMode:*Filesystem,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} Nov 24 23:27:12.982: FAIL: Unexpected error: <*errors.errorString | 0xc00242b830>: { s: "pod \"pod-51dcf2d0-d659-427d-81d0-d42a6e074dad\" is not Running: timed out waiting for the condition", } pod "pod-51dcf2d0-d659-427d-81d0-d42a6e074dad" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning(0x7973fe8, 0xc003ceb080, 0xc003a950a0, 0xc003d79180, 0xc003d79340, 0xc000d56900, 0x0, 0x0, 0x0, 0x0, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418 +0xe7d k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:198 +0x245 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000bdaf00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000bdaf00) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000bdaf00, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 24 23:27:12.983: INFO: deleting claim "provisioning-6376"/"pvc-m7cnr" Nov 24 23:27:13.014: INFO: deleting storage class provisioning-6376-e2e-scfxkk9 [AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "provisioning-6376". �[1mSTEP�[0m: Found 4 events. Nov 24 23:27:13.075: INFO: At 2022-11-24 23:22:12 +0000 UTC - event for pvc-m7cnr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 24 23:27:13.075: INFO: At 2022-11-24 23:22:12 +0000 UTC - event for pvc-m7cnr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 24 23:27:13.075: INFO: At 2022-11-24 23:22:12 +0000 UTC - event for pvc-m7cnr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } Provisioning: External provisioner is provisioning volume for claim "provisioning-6376/pvc-m7cnr" Nov 24 23:27:13.075: INFO: At 2022-11-24 23:22:22 +0000 UTC - event for pvc-m7cnr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-6v4kd_bc9b0e91-9736-40ff-b02c-9dc620bdda94 } ProvisioningFailed: failed to provision volume with StorageClass "provisioning-6376-e2e-scfxkk9": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 24 23:27:13.105: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 23:27:13.105: INFO: pod-51dcf2d0-d659-427d-81d0-d42a6e074dad Pending [] Nov 24 23:27:13.105: INFO: Nov 24 23:27:13.136: INFO: Logging node info for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:27:13.166: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-207.us-east-2.compute.internal 835a2849-50af-402f-ada2-dc2aaabf9491 7955 0 2022-11-24 23:15:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-207.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-057bc32a24e3000cb"} io.cilium.network.ipv4-cilium-host:172.20.35.125 io.cilium.network.ipv4-health-ip:172.20.49.16 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 23:15:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-24 23:15:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-24 23:15:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:16:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-24 23:17:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:17:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-057bc32a24e3000cb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:21 +0000 UTC,LastTransitionTime:2022-11-24 23:17:21 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:15:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:22:46 +0000 UTC,LastTransitionTime:2022-11-24 23:16:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.207,},NodeAddress{Type:ExternalIP,Address:3.145.191.93,},NodeAddress{Type:Hostname,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-207.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-191-93.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2099d5940b7e3a4ba5841af0a59dfe,SystemUUID:ec2099d5-940b-7e3a-4ba5-841af0a59dfe,BootID:c9acee07-6f8d-4ce4-9f64-0b9e72db9bae,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306305,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.1],SizeBytes:42900005,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.1],SizeBytes:41478143,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.1],SizeBytes:4955334,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:27:13.166: INFO: Logging kubelet events for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:27:13.199: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:27:13.270: INFO: cilium-operator-587dd66f87-5tzp7 started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container cilium-operator ready: true, restart count 1 Nov 24 23:27:13.270: INFO: ebs-csi-controller-764b64d679-6v4kd started at 2022-11-24 23:15:53 +0000 UTC (0+5 container statuses recorded) Nov 24 23:27:13.270: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:27:13.270: INFO: kube-apiserver-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+2 container statuses recorded) Nov 24 23:27:13.270: INFO: Container healthcheck ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 23:27:13.270: INFO: kube-scheduler-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container kube-scheduler ready: true, restart count 0 Nov 24 23:27:13.270: INFO: ebs-csi-node-cqt54 started at 2022-11-24 23:15:51 +0000 UTC (0+3 container statuses recorded) Nov 24 23:27:13.270: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:27:13.270: INFO: cilium-4p8l4 started at 2022-11-24 23:15:48 +0000 UTC (1+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:27:13.270: INFO: Container cilium-agent ready: true, restart count 1 Nov 24 23:27:13.270: INFO: dns-controller-845449865f-5vzhr started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container dns-controller ready: true, restart count 0 Nov 24 23:27:13.270: INFO: kops-controller-f99fk started at 2022-11-24 23:15:52 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container kops-controller ready: true, restart count 0 Nov 24 23:27:13.270: INFO: etcd-manager-events-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:27:13.270: INFO: etcd-manager-main-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container etcd-manager ready: true, restart count 0 Nov 24 23:27:13.270: INFO: kube-controller-manager-ip-172-20-33-207.us-east-2.compute.internal started at 2022-11-24 23:14:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.270: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 24 23:27:13.425: INFO: Latency metrics for node ip-172-20-33-207.us-east-2.compute.internal Nov 24 23:27:13.425: INFO: Logging node info for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:27:13.458: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-41-205.us-east-2.compute.internal b57c124d-0e36-4343-8be9-e4a6a30db6f0 14177 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-41-205.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-41-205.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eac761674c2bfd4d"} io.cilium.network.ipv4-cilium-host:172.20.37.165 io.cilium.network.ipv4-health-ip:172.20.58.34 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 23:22:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0eac761674c2bfd4d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:26:36 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.205,},NodeAddress{Type:ExternalIP,Address:18.118.7.188,},NodeAddress{Type:Hostname,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-41-205.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-118-7-188.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a0825c6cdcc6946eda61a19ac48ae,SystemUUID:ec2a0825-c6cd-cc69-46ed-a61a19ac48ae,BootID:ef6cc7ce-3768-4e2c-87b6-483e29d6ed0b,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 23:27:13.458: INFO: Logging kubelet events for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:27:13.492: INFO: Logging pods the kubelet thinks is on node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:27:13.526: INFO: hostexec-ip-172-20-41-205.us-east-2.compute.internal-lvbtg started at 2022-11-24 23:27:09 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 23:27:13.526: INFO: pod-exec-websocket-855996c2-1be2-439b-b2c2-d9db0f99c6b6 started at 2022-11-24 23:26:36 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container main ready: true, restart count 0 Nov 24 23:27:13.526: INFO: hostexec-ip-172-20-41-205.us-east-2.compute.internal-trrrj started at 2022-11-24 23:27:03 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:27:13.526: INFO: ebs-csi-node-wfbr5 started at 2022-11-24 23:17:06 +0000 UTC (0+3 container statuses recorded) Nov 24 23:27:13.526: INFO: Container ebs-plugin ready: true, restart count 0 Nov 24 23:27:13.526: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 23:27:13.526: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 23:27:13.526: INFO: busybox-ff078305-a49e-4a13-9ecb-ea964e7b4f5e started at 2022-11-24 23:26:46 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container busybox ready: true, restart count 0 Nov 24 23:27:13.526: INFO: httpd started at 2022-11-24 23:26:56 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container httpd ready: true, restart count 0 Nov 24 23:27:13.526: INFO: concurrent-27822207-q7kzk started at 2022-11-24 23:27:00 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container c ready: false, restart count 0 Nov 24 23:27:13.526: INFO: pod-projected-configmaps-1cbd4a7c-7ed0-4a48-8b65-90d6489d8c89 started at 2022-11-24 23:25:47 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 23:27:13.526: INFO: send-events-1823ab19-7762-4837-a313-917ad3764286 started at 2022-11-24 23:26:49 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.526: INFO: Container p ready: true, restart count 0 Nov 24 23:27:13.526: INFO: emptydir-injector started at 2022-11-24 23:27:03 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.527: INFO: Container emptydir-injector ready: false, restart count 0 Nov 24 23:27:13.527: INFO: failure-1 started at 2022-11-24 23:27:08 +0000 UTC (0+1 container statuses recorded) Nov 24 23:27:13.527: INFO: Container failure-1 ready: false, restart count 0 Nov 24 23:27:13.527: INFO: pod-service-account-5d8db43c-e468-4a54-bd5c-19126313114c started at <nil> (0+0 container statuses recorded) Nov 24 23:27:13.527: INFO: cilium-865h8 started at 2022-11-24 23:17:06 +0000 UTC (1+1 container statuses recorded) Nov 24 23:27:13.527: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 24 23:27:13.527: INFO: Container cilium-agent ready: true, restart count 0 Nov 24 23:27:13.748: INFO: Latency metrics for node ip-172-20-41-205.us-east-2.compute.internal Nov 24 23:27:13.748: INFO: Logging node info for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:27:13.778: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-101.us-east-2.compute.internal 57616229-41ca-45ea-a9c0-62a480f79cd3 15615 0 2022-11-24 23:17:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-101.us-east-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:ip-172-20-43-101.us-east-2.compute.internal topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6460":"ip-172-20-43-101.us-east-2.compute.internal","ebs.csi.aws.com":"i-08a98f81ee270982c"} io.cilium.network.ipv4-cilium-host:172.20.54.43 io.cilium.network.ipv4-health-ip:172.20.34.140 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 23:17:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-24 23:17:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-24 23:17:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 23:26:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 23:26:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-08a98f81ee270982c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 23:17:31 +0000 UTC,LastTransitionTime:2022-11-24 23:17:31 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 23:26:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 23:26:57 +0000 UTC,LastTransitionTime:2022-11-24 23:17:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.101,},NodeAddress{Type:ExternalIP,Address:3.145.115.218,},NodeAddress{Type:Hostname,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-101.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-145-115-218.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255f229f75816729a88b9360bb2edc,SystemUUID:ec255f22-9f75-8167-29a8-8b9360bb2edc,BootID:42ef11d7-09e9-4184-b34d-650182684646,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6460^758025c2-6c4f-11ed-bebc-2a2dfedf8700],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6460^758025c2-6c4f-11ed-bebc-2a2dfedf8700,DevicePath:,},},Config:nil,},} Nov 24 23:27:13.779: INFO: Logging kubelet events for node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:27:13.810: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-101.us-east-2.compute.internal Nov 24 23:27:13.843: INFO: cilium-nv9cv started at 2022-11-24 23:17:07 +0000 UTC (1+1 container statuses recorded) Nov 24 23:27:13.843: INFO: Init container clean-cilium-state ready: true