Result | FAILURE |
Tests | 69 failed / 698 succeeded |
Started | |
Elapsed | 57m43s |
Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 28 23:44:32.183: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc002c326d0>: { s: "pod \"pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883\" is not Running: timed out waiting for the condition", } pod "pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_09.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:39:31.372: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 28 23:39:31.703: INFO: Creating resource for dynamic PV Nov 28 23:39:31.703: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-8260-e2e-sc7vlbl �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 28 23:44:32.183: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc002c326d0>: { s: "pod \"pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883\" is not Running: timed out waiting for the condition", } pod "pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00046b980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00046b980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00046b980, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 28 23:44:32.183: INFO: Deleting pod "pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883" in namespace "volume-expand-8260" Nov 28 23:44:32.252: INFO: Wait up to 5m0s for pod "pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 28 23:44:32.315: INFO: Deleting pod "pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883" in namespace "volume-expand-8260" �[1mSTEP�[0m: Deleting pvc Nov 28 23:44:32.445: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com2kv6s" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-8260". �[1mSTEP�[0m: Found 5 events. Nov 28 23:44:32.638: INFO: At 2022-11-28 23:39:31 +0000 UTC - event for ebs.csi.aws.com2kv6s: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:44:32.638: INFO: At 2022-11-28 23:39:31 +0000 UTC - event for ebs.csi.aws.com2kv6s: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:44:32.638: INFO: At 2022-11-28 23:39:31 +0000 UTC - event for ebs.csi.aws.com2kv6s: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "volume-expand-8260/ebs.csi.aws.com2kv6s" Nov 28 23:44:32.638: INFO: At 2022-11-28 23:39:42 +0000 UTC - event for ebs.csi.aws.com2kv6s: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8260-e2e-sc7vlbl": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:44:32.638: INFO: At 2022-11-28 23:41:03 +0000 UTC - event for ebs.csi.aws.com2kv6s: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8260-e2e-sc7vlbl": rpc error: code = Internal desc = Could not create volume "pvc-87f0bec2-3f4d-4e63-b1fb-b51ff48da36c": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:44:32.701: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:44:32.701: INFO: Nov 28 23:44:32.766: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:32.831: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 34218 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:41:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:32.831: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:32.897: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:32.964: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:32.964: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:32.964: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:32.964: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:32.964: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container coredns ready: true, restart count 0 Nov 28 23:44:32.964: INFO: netserver-0 started at 2022-11-28 23:44:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:32.964: INFO: test-rolling-update-with-lb-7db9cf5ff8-bjp4c started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:32.964: INFO: startup-adb08a1a-8594-4c57-8938-88cc08e1a18f started at 2022-11-28 23:41:11 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container busybox ready: false, restart count 0 Nov 28 23:44:32.964: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:32.964: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:32.964: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:44:32.964: INFO: netserver-0 started at 2022-11-28 23:44:04 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:32.964: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-z4hp8 started at 2022-11-28 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.964: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:44:33.247: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:33.247: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:33.311: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 38659 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:33.311: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:33.378: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:33.450: INFO: send-events-f1bc8d10-fa38-4e63-97eb-6a82dd4283f9 started at 2022-11-28 23:44:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Container p ready: false, restart count 0 Nov 28 23:44:33.450: INFO: test-container-pod started at 2022-11-28 23:44:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Container webserver ready: false, restart count 0 Nov 28 23:44:33.450: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:33.450: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:33.450: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:33.450: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:33.450: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:33.450: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:33.450: INFO: test-rolling-update-with-lb-7db9cf5ff8-fd9gb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:33.450: INFO: netserver-1 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:33.450: INFO: netserver-1 started at 2022-11-28 23:44:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:33.450: INFO: condition-test-b68v6 started at 2022-11-28 23:44:28 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.450: INFO: Container httpd ready: true, restart count 0 Nov 28 23:44:33.735: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:33.735: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:33.799: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 38663 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-217":"ip-172-20-51-62.us-west-2.compute.internal","ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:44:14 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:44:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-217^8ff49ff7-6f76-11ed-af79-e248a53deb4d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-217^8ff49ff7-6f76-11ed-af79-e248a53deb4d,DevicePath:,},},Config:nil,},} Nov 28 23:44:33.800: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:33.865: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:33.932: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:33.932: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:33.932: INFO: netserver-2 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.932: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:33.932: INFO: netserver-2 started at 2022-11-28 23:44:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.932: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:33.932: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:33.932: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:33.932: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:44:08 +0000 UTC (0+7 container statuses recorded) Nov 28 23:44:33.932: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:33.932: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:34.215: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:34.215: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:34.279: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 37644 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:41:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f,DevicePath:,},},Config:nil,},} Nov 28 23:44:34.281: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:34.354: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:34.421: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:34.421: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:34.422: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:34.422: INFO: test-container-pod started at 2022-11-28 23:44:27 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.422: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:34.422: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:34.422: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:34.422: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:34.422: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:34.422: INFO: test-rolling-update-with-lb-7db9cf5ff8-s9xpb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.422: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:34.422: INFO: startup-7d72699d-d039-4dca-bdad-05b0cfd4003f started at 2022-11-28 23:43:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.422: INFO: Container busybox ready: true, restart count 0 Nov 28 23:44:34.422: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.422: INFO: Container coredns ready: true, restart count 0 Nov 28 23:44:34.422: INFO: netserver-3 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.422: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:34.422: INFO: netserver-3 started at 2022-11-28 23:44:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.422: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:34.691: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:34.691: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:34.757: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 34762 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:34.758: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:34.829: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:34.897: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.897: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:44:34.897: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.897: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:44:34.897: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:44:34.897: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:44:34.897: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:44:34.897: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.897: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:44:34.897: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:34.897: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:34.897: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:34.897: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:34.897: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:34.897: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:34.897: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:34.897: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.898: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:44:34.898: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:44:34.898: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:44:34.898: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:44:34.898: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:44:34.898: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:34.898: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:34.898: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.898: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:44:34.898: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.898: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:44:34.898: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:34.898: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:44:35.221: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:35.221: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-8260" for this suite.
Find pod-9dc8b990-d3b3-48fc-b7aa-78f73f610883 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 28 23:49:58.855: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003cc2b70>: { s: "pod \"pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2\" is not Running: timed out waiting for the condition", } pod "pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_07.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:44:58.104: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 28 23:44:58.416: INFO: Creating resource for dynamic PV Nov 28 23:44:58.416: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-5866-e2e-sczqlqr �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 28 23:49:58.855: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc003cc2b70>: { s: "pod \"pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2\" is not Running: timed out waiting for the condition", } pod "pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00046fc80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00046fc80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00046fc80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 28 23:49:58.856: INFO: Deleting pod "pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2" in namespace "volume-expand-5866" Nov 28 23:49:58.925: INFO: Wait up to 5m0s for pod "pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 28 23:49:58.987: INFO: Deleting pod "pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2" in namespace "volume-expand-5866" �[1mSTEP�[0m: Deleting pvc Nov 28 23:49:59.111: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.commcsmf" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-5866". �[1mSTEP�[0m: Found 4 events. Nov 28 23:49:59.300: INFO: At 2022-11-28 23:44:58 +0000 UTC - event for ebs.csi.aws.commcsmf: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:49:59.300: INFO: At 2022-11-28 23:44:58 +0000 UTC - event for ebs.csi.aws.commcsmf: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:49:59.300: INFO: At 2022-11-28 23:44:58 +0000 UTC - event for ebs.csi.aws.commcsmf: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "volume-expand-5866/ebs.csi.aws.commcsmf" Nov 28 23:49:59.300: INFO: At 2022-11-28 23:45:08 +0000 UTC - event for ebs.csi.aws.commcsmf: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-5866-e2e-sczqlqr": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:49:59.369: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:49:59.369: INFO: Nov 28 23:49:59.436: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:59.499: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 45759 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:48:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:49:59.499: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:59.563: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:59.630: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:49:59.630: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:49:59.630: INFO: pod-init-4cadbde1-9293-41bf-b984-f8f6a5d9dd5c started at 2022-11-28 23:49:09 +0000 UTC (2+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Init container init1 ready: false, restart count 3 Nov 28 23:49:59.630: INFO: Init container init2 ready: false, restart count 0 Nov 28 23:49:59.630: INFO: Container run1 ready: false, restart count 0 Nov 28 23:49:59.630: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:49:59.630: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-lt6pr started at 2022-11-28 23:49:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:49:59.630: INFO: fail-once-local-6qnsw started at 2022-11-28 23:49:58 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Container c ready: false, restart count 0 Nov 28 23:49:59.630: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:49:59.630: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:59.630: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:59.630: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:59.630: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Container coredns ready: true, restart count 0 Nov 28 23:49:59.630: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-4p7zj started at 2022-11-28 23:49:54 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:59.630: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:49:59.964: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:59.964: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:50:00.029: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 44966 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:50:00.030: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:50:00.094: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:50:00.163: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:00.163: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:50:00.163: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:50:00.163: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:50:00.163: INFO: pod-projected-configmaps-4cef6045-0f23-4207-9299-ff442791dae9 started at 2022-11-28 23:49:39 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:00.163: INFO: Container createcm-volume-test ready: true, restart count 0 Nov 28 23:50:00.163: INFO: Container delcm-volume-test ready: true, restart count 0 Nov 28 23:50:00.163: INFO: Container updcm-volume-test ready: true, restart count 0 Nov 28 23:50:00.163: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:50:00.163: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:50:00.163: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:50:00.163: INFO: concurrent-27827990-jlnt4 started at <nil> (0+0 container statuses recorded) Nov 28 23:50:00.431: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:50:00.431: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:50:00.493: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 41438 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:50:00.494: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:50:00.563: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:50:00.640: INFO: fail-once-local-ql69d started at 2022-11-28 23:49:58 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Container c ready: false, restart count 0 Nov 28 23:50:00.640: INFO: webserver-pod started at 2022-11-28 23:46:39 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Container agnhost ready: false, restart count 0 Nov 28 23:50:00.640: INFO: test-webserver-5baf5b03-7bc7-4db6-b317-09e6eb20d71d started at 2022-11-28 23:48:52 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Container test-webserver ready: true, restart count 0 Nov 28 23:50:00.640: INFO: downwardapi-volume-f6723a62-f013-4335-8e57-1648b46842db started at 2022-11-28 23:49:56 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Container client-container ready: false, restart count 0 Nov 28 23:50:00.640: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:00.640: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:50:00.640: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:50:00.640: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:50:00.640: INFO: hostexec-ip-172-20-51-62.us-west-2.compute.internal-wj7ts started at 2022-11-28 23:49:52 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:50:00.640: INFO: pod-ea89ef8a-e7ce-4e11-90b3-577789d5305a started at 2022-11-28 23:49:59 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Container write-pod ready: false, restart count 0 Nov 28 23:50:00.640: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:50:00.640: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:50:00.640: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:50:01.268: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:50:01.268: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:50:01.331: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 46075 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1936":"csi-mock-csi-mock-volumes-1936","ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2022-11-28 23:49:51 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:54 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:54 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:54 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:49:54 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f kubernetes.io/csi/csi-mock-csi-mock-volumes-1936^4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1936^4,DevicePath:,},},Config:nil,},} Nov 28 23:50:01.331: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:50:01.396: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:50:01.462: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.462: INFO: Container coredns ready: true, restart count 0 Nov 28 23:50:01.462: INFO: dns-test-da32e99d-1de1-42e2-8356-573ac5402bed started at 2022-11-28 23:49:35 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:01.462: INFO: Container jessie-querier ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container querier ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container webserver ready: true, restart count 0 Nov 28 23:50:01.462: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:50:01.462: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:50:01.462: INFO: csi-mockplugin-0 started at 2022-11-28 23:49:45 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:01.462: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container mock ready: true, restart count 0 Nov 28 23:50:01.462: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:01.462: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:50:01.462: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:50:01.462: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:49:45 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.462: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:50:01.775: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:50:01.775: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:50:01.838: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 42158 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:50:01.838: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:50:01.903: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:50:01.972: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:50:01.973: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:50:01.973: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:50:01.973: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:50:01.973: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:50:01.973: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:50:01.973: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:50:01.973: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:50:01.973: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:50:01.973: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:50:01.973: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:50:01.973: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:50:01.973: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:50:01.973: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:50:01.973: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:50:01.973: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:50:02.225: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:50:02.226: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-5866" for this suite.
Find pod-3dde04f4-9f62-40f2-b89e-520c415ea7a2 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 28 23:39:24.220: Unexpected error: <*errors.errorString | 0xc000242280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385from junit_13.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:34:23.385: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volumemode �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should not mount / map unused volumes in a pod [LinuxOnly] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Nov 28 23:34:23.751: INFO: Creating resource for dynamic PV Nov 28 23:34:23.751: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volumemode-9520-e2e-sclsdng �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating pod Nov 28 23:39:24.220: FAIL: Unexpected error: <*errors.errorString | 0xc000242280>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385 +0x6c5 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000a37c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000a37c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000a37c80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 28 23:39:24.220: INFO: Deleting pod "pod-4885531d-4e0f-4975-b2e6-c9e20183f774" in namespace "volumemode-9520" Nov 28 23:39:24.288: INFO: Wait up to 5m0s for pod "pod-4885531d-4e0f-4975-b2e6-c9e20183f774" to be fully deleted �[1mSTEP�[0m: Deleting pvc Nov 28 23:39:24.416: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com7zq49" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volumemode-9520". �[1mSTEP�[0m: Found 5 events. Nov 28 23:39:24.610: INFO: At 2022-11-28 23:34:23 +0000 UTC - event for ebs.csi.aws.com7zq49: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:39:24.610: INFO: At 2022-11-28 23:34:24 +0000 UTC - event for ebs.csi.aws.com7zq49: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:39:24.610: INFO: At 2022-11-28 23:34:24 +0000 UTC - event for ebs.csi.aws.com7zq49: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "volumemode-9520/ebs.csi.aws.com7zq49" Nov 28 23:39:24.610: INFO: At 2022-11-28 23:34:34 +0000 UTC - event for ebs.csi.aws.com7zq49: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-9520-e2e-sclsdng": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:39:24.610: INFO: At 2022-11-28 23:35:29 +0000 UTC - event for ebs.csi.aws.com7zq49: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volumemode-9520-e2e-sclsdng": rpc error: code = Internal desc = Could not create volume "pvc-734a8502-6ef0-4105-9b34-9035ca5b8a4d": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:39:24.674: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:39:24.674: INFO: Nov 28 23:39:24.738: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:24.802: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 25977 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:38:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:24.802: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:24.868: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:24.938: INFO: my-hostname-basic-85e084c4-0ff0-4741-a3ff-25529682c210-2dtxg started at 2022-11-28 23:39:23 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container my-hostname-basic-85e084c4-0ff0-4741-a3ff-25529682c210 ready: false, restart count 0 Nov 28 23:39:24.938: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:24.938: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:24.938: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:39:24.938: INFO: execpodm99vr started at 2022-11-28 23:39:18 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:39:24.938: INFO: busybox-scheduling-5968efa6-3643-4b09-9605-4298963b5725 started at 2022-11-28 23:39:12 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container busybox-scheduling-5968efa6-3643-4b09-9605-4298963b5725 ready: true, restart count 0 Nov 28 23:39:24.938: INFO: annotationupdate58e6a497-5ca8-4c43-a391-4c94b518b80f started at 2022-11-28 23:39:23 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container client-container ready: false, restart count 0 Nov 28 23:39:24.938: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container coredns ready: true, restart count 0 Nov 28 23:39:24.938: INFO: pod-subpath-test-secret-ctrd started at 2022-11-28 23:39:07 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container test-container-subpath-secret-ctrd ready: true, restart count 0 Nov 28 23:39:24.938: INFO: pod-submit-remove-16785b9b-13f7-4576-8e61-3d076d4136c2 started at 2022-11-28 23:39:21 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container nginx ready: true, restart count 0 Nov 28 23:39:24.938: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:24.938: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:24.938: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:24.938: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:24.938: INFO: startup-19549d57-ef5e-4e10-901b-65ce5445ea92 started at 2022-11-28 23:39:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:24.938: INFO: Container busybox ready: false, restart count 0 Nov 28 23:39:25.235: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:25.235: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:25.300: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 26005 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8286":"csi-mock-csi-mock-volumes-8286","ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:35:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311,DevicePath:,},},Config:nil,},} Nov 28 23:39:25.300: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:25.366: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:25.434: INFO: webhook-to-be-mutated started at 2022-11-28 23:38:29 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Init container webhook-added-init-container ready: false, restart count 0 Nov 28 23:39:25.434: INFO: Container example ready: false, restart count 0 Nov 28 23:39:25.434: INFO: pod-subpath-test-inlinevolume-r9rj started at 2022-11-28 23:39:15 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Init container init-volume-inlinevolume-r9rj ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container test-container-subpath-inlinevolume-r9rj ready: true, restart count 0 Nov 28 23:39:25.434: INFO: csi-mockplugin-0 started at 2022-11-28 23:37:31 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:25.434: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container mock ready: true, restart count 0 Nov 28 23:39:25.434: INFO: startup-script started at 2022-11-28 23:38:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Container startup-script ready: true, restart count 0 Nov 28 23:39:25.434: INFO: hostexec-ip-172-20-43-89.us-west-2.compute.internal-jpffs started at 2022-11-28 23:39:17 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:39:25.434: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:37:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:39:25.434: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:25.434: INFO: pvc-volume-tester-hj2f9 started at 2022-11-28 23:37:41 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Container volume-tester ready: true, restart count 0 Nov 28 23:39:25.434: INFO: pod-server-1 started at 2022-11-28 23:39:04 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Init container init ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:39:25.434: INFO: pod-subpath-test-configmap-pjcd started at 2022-11-28 23:39:04 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.434: INFO: Container test-container-subpath-configmap-pjcd ready: true, restart count 0 Nov 28 23:39:25.434: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:25.434: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:25.434: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:25.713: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:25.713: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:25.777: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 23138 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:34:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:25.777: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:25.843: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:25.910: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:25.910: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:25.910: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:25.910: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:25.910: INFO: busybox-0d1169e6-d555-4838-9ecb-a54946f6911e started at 2022-11-28 23:38:54 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.910: INFO: Container busybox ready: true, restart count 0 Nov 28 23:39:25.910: INFO: httpd started at 2022-11-28 23:39:18 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.910: INFO: Container httpd ready: false, restart count 0 Nov 28 23:39:25.910: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:25.910: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:25.910: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:25.910: INFO: pod-client started at 2022-11-28 23:39:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.910: INFO: Container pod-client ready: true, restart count 0 Nov 28 23:39:25.910: INFO: deployment-shared-map-item-removal-55649fd747-9v85c started at 2022-11-28 23:38:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.910: INFO: Container nginx ready: false, restart count 0 Nov 28 23:39:25.910: INFO: inline-volume-vhkw9 started at 2022-11-28 23:37:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:25.910: INFO: Container volume-tester ready: false, restart count 0 Nov 28 23:39:26.159: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:26.159: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:26.222: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 26862 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:39:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:26.223: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:26.296: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:26.364: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container coredns ready: true, restart count 0 Nov 28 23:39:26.364: INFO: fail-once-non-local-n24jt started at 2022-11-28 23:39:12 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container c ready: false, restart count 0 Nov 28 23:39:26.364: INFO: fail-once-non-local-7bkhh started at 2022-11-28 23:39:12 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container c ready: false, restart count 0 Nov 28 23:39:26.364: INFO: deployment-shared-map-item-removal-55649fd747-bgf96 started at 2022-11-28 23:38:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container nginx ready: false, restart count 0 Nov 28 23:39:26.364: INFO: labelsupdatec1328a21-c527-4e3f-8d1c-1f0896e76e55 started at 2022-11-28 23:39:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container client-container ready: false, restart count 0 Nov 28 23:39:26.364: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:26.364: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:26.364: INFO: fail-once-non-local-cmswk started at 2022-11-28 23:39:17 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container c ready: false, restart count 0 Nov 28 23:39:26.364: INFO: csi-mockplugin-0 started at 2022-11-28 23:38:33 +0000 UTC (0+4 container statuses recorded) Nov 28 23:39:26.364: INFO: Container busybox ready: true, restart count 0 Nov 28 23:39:26.364: INFO: Container csi-provisioner ready: false, restart count 1 Nov 28 23:39:26.364: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:39:26.364: INFO: Container mock ready: true, restart count 0 Nov 28 23:39:26.364: INFO: fail-once-non-local-d7ns4 started at 2022-11-28 23:39:22 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container c ready: false, restart count 0 Nov 28 23:39:26.364: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:26.364: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:26.364: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:26.364: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:26.364: INFO: fail-once-non-local-r67bp started at 2022-11-28 23:39:17 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.364: INFO: Container c ready: false, restart count 0 Nov 28 23:39:26.634: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:26.634: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:26.698: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 22365 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:26.698: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:26.765: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:26.833: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:39:26.833: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:39:26.833: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:39:26.833: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:26.833: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:26.833: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:39:26.833: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:39:26.833: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:39:26.833: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:39:26.833: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:39:26.833: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:26.833: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:26.833: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:39:26.833: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:39:26.833: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:26.833: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:27.090: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:27.090: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volumemode-9520" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumes\sshould\sstore\sdata$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 28 23:44:36.766: Failed to create injector pod: timed out waiting for the condition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186from junit_10.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:39:35.841: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should store data /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159 Nov 28 23:39:36.170: INFO: Creating resource for dynamic PV Nov 28 23:39:36.170: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volume-7013-e2e-sc64j8l �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: starting external-injector �[1mSTEP�[0m: Deleting pod external-injector in namespace volume-7013 Nov 28 23:44:36.702: INFO: Waiting for pod external-injector to disappear Nov 28 23:44:36.766: INFO: Pod external-injector no longer exists Nov 28 23:44:36.766: FAIL: Failed to create injector pod: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumesTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:186 +0x3ff k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000322a80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000322a80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000322a80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: cleaning the environment after external �[1mSTEP�[0m: Deleting pvc Nov 28 23:44:36.831: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comdfvl5" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-7013". �[1mSTEP�[0m: Found 4 events. Nov 28 23:44:37.024: INFO: At 2022-11-28 23:39:36 +0000 UTC - event for ebs.csi.aws.comdfvl5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:44:37.024: INFO: At 2022-11-28 23:39:36 +0000 UTC - event for ebs.csi.aws.comdfvl5: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:44:37.024: INFO: At 2022-11-28 23:39:36 +0000 UTC - event for ebs.csi.aws.comdfvl5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "volume-7013/ebs.csi.aws.comdfvl5" Nov 28 23:44:37.024: INFO: At 2022-11-28 23:39:46 +0000 UTC - event for ebs.csi.aws.comdfvl5: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-7013-e2e-sc64j8l": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:44:37.088: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:44:37.088: INFO: Nov 28 23:44:37.152: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:37.216: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 34218 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:41:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:37.216: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:37.283: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:37.351: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:44:37.351: INFO: netserver-0 started at 2022-11-28 23:44:04 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:37.351: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-z4hp8 started at 2022-11-28 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:44:37.351: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-7xxjz started at 2022-11-28 23:44:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:44:37.351: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:37.351: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:37.351: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:37.351: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:37.351: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container coredns ready: true, restart count 0 Nov 28 23:44:37.351: INFO: netserver-0 started at 2022-11-28 23:44:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:37.351: INFO: test-rolling-update-with-lb-7db9cf5ff8-bjp4c started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:37.351: INFO: startup-adb08a1a-8594-4c57-8938-88cc08e1a18f started at 2022-11-28 23:41:11 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Container busybox ready: false, restart count 0 Nov 28 23:44:37.351: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:37.351: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:37.351: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:37.678: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:37.678: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:37.742: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 38659 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:37.742: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:37.808: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:37.875: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:37.875: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:37.875: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:37.875: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:37.875: INFO: test-rolling-update-with-lb-7db9cf5ff8-fd9gb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.875: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:37.875: INFO: netserver-1 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.875: INFO: Container webserver ready: false, restart count 0 Nov 28 23:44:37.875: INFO: netserver-1 started at 2022-11-28 23:44:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.875: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:37.875: INFO: send-events-f1bc8d10-fa38-4e63-97eb-6a82dd4283f9 started at 2022-11-28 23:44:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.875: INFO: Container p ready: true, restart count 0 Nov 28 23:44:37.875: INFO: test-container-pod started at 2022-11-28 23:44:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:37.875: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:37.875: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:37.875: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:37.875: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:38.209: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:38.209: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:38.275: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 39040 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-217":"ip-172-20-51-62.us-west-2.compute.internal","ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:38.276: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:38.342: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:38.410: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:38.410: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:38.410: INFO: netserver-2 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.410: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:38.410: INFO: netserver-2 started at 2022-11-28 23:44:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.410: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:38.410: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:38.410: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:38.410: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:44:08 +0000 UTC (0+7 container statuses recorded) Nov 28 23:44:38.410: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:38.410: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:38.700: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:38.700: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:38.763: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 37644 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:41:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f,DevicePath:,},},Config:nil,},} Nov 28 23:44:38.765: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:38.837: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:38.904: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.904: INFO: Container coredns ready: true, restart count 0 Nov 28 23:44:38.904: INFO: netserver-3 started at 2022-11-28 23:44:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.904: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:38.904: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:38.904: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:38.904: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:38.904: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:38.904: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:38.904: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:38.904: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:38.904: INFO: test-rolling-update-with-lb-7db9cf5ff8-s9xpb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.904: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:38.904: INFO: pod-projected-secrets-6bed42bd-73a0-453f-8df5-7da2bf05de12 started at 2022-11-28 23:44:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.904: INFO: Container projected-secret-volume-test ready: false, restart count 0 Nov 28 23:44:38.904: INFO: startup-7d72699d-d039-4dca-bdad-05b0cfd4003f started at 2022-11-28 23:43:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:38.904: INFO: Container busybox ready: true, restart count 0 Nov 28 23:44:39.171: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:39.171: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:39.237: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 34762 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:39.238: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:39.306: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:39.382: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:44:39.382: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:44:39.382: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:39.382: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:44:39.382: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:44:39.382: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:39.382: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:44:39.382: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:44:39.382: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:39.382: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:39.382: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:44:39.382: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:39.382: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:44:39.382: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:44:39.382: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:44:39.382: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:44:39.678: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:44:39.678: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-7013" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 28 23:27:23.338: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc0041f2fd0>: { s: "pod \"pod-99828920-35f1-4eb5-894d-226e6e6b50cf\" is not Running: timed out waiting for the condition", } pod "pod-99828920-35f1-4eb5-894d-226e6e6b50cf" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_05.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:22:22.534: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Nov 28 23:22:22.864: INFO: Creating resource for dynamic PV Nov 28 23:22:22.864: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-8707-e2e-scvhqqt �[1mSTEP�[0m: creating a claim Nov 28 23:22:22.932: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 28 23:27:23.338: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc0041f2fd0>: { s: "pod \"pod-99828920-35f1-4eb5-894d-226e6e6b50cf\" is not Running: timed out waiting for the condition", } pod "pod-99828920-35f1-4eb5-894d-226e6e6b50cf" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x34d k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000bea600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000bea600) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000bea600, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 28 23:27:23.338: INFO: Deleting pod "pod-99828920-35f1-4eb5-894d-226e6e6b50cf" in namespace "volume-expand-8707" Nov 28 23:27:23.407: INFO: Wait up to 5m0s for pod "pod-99828920-35f1-4eb5-894d-226e6e6b50cf" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 28 23:27:23.472: INFO: Deleting pod "pod-99828920-35f1-4eb5-894d-226e6e6b50cf" in namespace "volume-expand-8707" �[1mSTEP�[0m: Deleting pvc Nov 28 23:27:23.603: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com9k2pr" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-8707". �[1mSTEP�[0m: Found 5 events. Nov 28 23:27:23.803: INFO: At 2022-11-28 23:22:22 +0000 UTC - event for ebs.csi.aws.com9k2pr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:27:23.803: INFO: At 2022-11-28 23:22:23 +0000 UTC - event for ebs.csi.aws.com9k2pr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:27:23.803: INFO: At 2022-11-28 23:22:23 +0000 UTC - event for ebs.csi.aws.com9k2pr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "volume-expand-8707/ebs.csi.aws.com9k2pr" Nov 28 23:27:23.803: INFO: At 2022-11-28 23:22:33 +0000 UTC - event for ebs.csi.aws.com9k2pr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8707-e2e-scvhqqt": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:27:23.803: INFO: At 2022-11-28 23:23:28 +0000 UTC - event for ebs.csi.aws.com9k2pr: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-8707-e2e-scvhqqt": rpc error: code = Internal desc = Could not create volume "pvc-22e18283-0c35-44aa-b95d-f44d70f6bfa1": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:27:23.869: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:27:23.869: INFO: Nov 28 23:27:23.937: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:27:24.002: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 8255 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-4714":"ip-172-20-33-9.us-west-2.compute.internal","ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:20:28 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:20:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:26:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-4714^3da39203-6f73-11ed-9ff4-dac3ce5938e2],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-4714^3da39203-6f73-11ed-9ff4-dac3ce5938e2,DevicePath:,},},Config:nil,},} Nov 28 23:27:24.003: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:27:24.071: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:27:24.214: INFO: pod-4b13b367-9436-447a-9389-ba41e0e58746 started at 2022-11-28 23:20:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container write-pod ready: false, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-jrjjf started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-dd5fh started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-6t57g started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:27:24.214: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:27:24.214: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:27:24.214: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:22:40 +0000 UTC (0+7 container statuses recorded) Nov 28 23:27:24.214: INFO: Container csi-attacher ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container csi-provisioner ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container csi-resizer ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container hostpath ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container liveness-probe ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 28 23:27:24.214: INFO: pod-subpath-test-inlinevolume-hn7q started at 2022-11-28 23:23:19 +0000 UTC (2+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Init container init-volume-inlinevolume-hn7q ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Init container test-init-volume-inlinevolume-hn7q ready: false, restart count 0 Nov 28 23:27:24.214: INFO: Container test-container-subpath-inlinevolume-hn7q ready: false, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-dstxj started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:20:00 +0000 UTC (0+7 container statuses recorded) Nov 28 23:27:24.214: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:24.214: INFO: pod-subpath-test-preprovisionedpv-4ldf started at 2022-11-28 23:26:25 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container test-container-subpath-preprovisionedpv-4ldf ready: false, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-r9k29 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-drjbg started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-qgkzt started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-tgb48 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-sbskj started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.214: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:27:24.214: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:24.214: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:24.214: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container coredns ready: true, restart count 0 Nov 28 23:27:24.214: INFO: pod-subpath-test-preprovisionedpv-w7dp started at 2022-11-28 23:22:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container test-container-subpath-preprovisionedpv-w7dp ready: false, restart count 0 Nov 28 23:27:24.214: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-h9dj2 started at 2022-11-28 23:22:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:27:24.214: INFO: netserver-0 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.214: INFO: Container webserver ready: true, restart count 0 Nov 28 23:27:24.215: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-277wv started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.215: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:27:24.215: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-z68b9 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.215: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.215: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-r7vgn started at 2022-11-28 23:27:14 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.215: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:27:24.531: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:27:24.531: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:27:24.597: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 8671 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-561":"ip-172-20-43-89.us-west-2.compute.internal","ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:26:34 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:26:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:26:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-561^1818f23b-6f74-11ed-a3cd-56570fd01eb8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-561^1818f23b-6f74-11ed-a3cd-56570fd01eb8,DevicePath:,},},Config:nil,},} Nov 28 23:27:24.598: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:27:24.666: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:27:24.740: INFO: termination-message-container5295c1ca-0d54-4b59-bfa1-69951bdc3eab started at 2022-11-28 23:24:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container termination-message-container ready: false, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-rw6kp started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-kzswp started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-p6vgv started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:27:24.740: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:26:30 +0000 UTC (0+7 container statuses recorded) Nov 28 23:27:24.740: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:24.740: INFO: inline-volume-tester-d7ss2 started at 2022-11-28 23:26:33 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 28 23:27:24.740: INFO: privileged-pod started at 2022-11-28 23:24:43 +0000 UTC (0+2 container statuses recorded) Nov 28 23:27:24.740: INFO: Container not-privileged-container ready: false, restart count 0 Nov 28 23:27:24.740: INFO: Container privileged-container ready: false, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-j5xkz started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: csi-mockplugin-0 started at 2022-11-28 23:26:41 +0000 UTC (0+3 container statuses recorded) Nov 28 23:27:24.740: INFO: Container csi-provisioner ready: false, restart count 0 Nov 28 23:27:24.740: INFO: Container driver-registrar ready: false, restart count 0 Nov 28 23:27:24.740: INFO: Container mock ready: false, restart count 0 Nov 28 23:27:24.740: INFO: netserver-1 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container webserver ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-qvrrq started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: downwardapi-volume-dbbd0dde-9320-4c6e-8d09-dc2f6caec1b0 started at 2022-11-28 23:25:07 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container client-container ready: false, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-xtzkx started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:27:24.740: INFO: busybox-df981fcf-34dd-4d33-972f-8fbe9a730ecc started at 2022-11-28 23:24:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container busybox ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-bxd52 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-flrbd started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:27:24.740: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:24.740: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-sdflf started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:24.740: INFO: busybox-user-65534-625e0c38-ec9d-4a82-9e0c-b7f9ab373147 started at 2022-11-28 23:23:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container busybox-user-65534-625e0c38-ec9d-4a82-9e0c-b7f9ab373147 ready: false, restart count 0 Nov 28 23:27:24.740: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-d9z4t started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:24.740: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.020: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:27:25.020: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:27:25.086: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 7270 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-28 23:22:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:27:25.086: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:27:25.155: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-zrxm7 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:27:25.227: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:27:25.227: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:25.227: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:27:25.227: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:27:25.227: INFO: pod-subpath-test-configmap-mq9r started at 2022-11-28 23:26:45 +0000 UTC (1+2 container statuses recorded) Nov 28 23:27:25.227: INFO: Init container init-volume-configmap-mq9r ready: true, restart count 0 Nov 28 23:27:25.227: INFO: Container test-container-subpath-configmap-mq9r ready: false, restart count 2 Nov 28 23:27:25.227: INFO: Container test-container-volume-configmap-mq9r ready: true, restart count 0 Nov 28 23:27:25.227: INFO: pod-8c9c80cf-23b4-4724-a382-813ae51001fa started at 2022-11-28 23:26:12 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container test-container ready: false, restart count 0 Nov 28 23:27:25.227: INFO: logs-generator started at 2022-11-28 23:24:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container logs-generator ready: false, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-q9dwk started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-9w68m started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-glk2d started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-s68jn started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-8kqzt started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: netserver-2 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container webserver ready: false, restart count 0 Nov 28 23:27:25.227: INFO: pod-projected-configmaps-00ae5f67-4f57-4c9f-8bb9-ac4dbcb2ad8e started at 2022-11-28 23:27:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container agnhost-container ready: false, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-xh7kh started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-4vt6f started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-hd7zs started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.227: INFO: pod-2965f384-9fea-4ce7-9043-b90b696f8e39 started at 2022-11-28 23:26:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container test-container ready: false, restart count 0 Nov 28 23:27:25.227: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-dzzdn started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.227: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.492: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:27:25.492: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:27:25.559: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 8899 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-28 23:26:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:53 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:53 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:26:53 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:26:53 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:27:25.559: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:27:25.627: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:27:25.698: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:27:25.698: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-sp6hj started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: netserver-3 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container webserver ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-jvz2t started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-747bf started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-rs47m started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-tlf72 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:27:25.698: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:27:25.698: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:25.698: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:25.698: INFO: hostexec-ip-172-20-59-24.us-west-2.compute.internal-59l8d started at 2022-11-28 23:26:59 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-7gg2s started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-khs7v started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container coredns ready: true, restart count 0 Nov 28 23:27:25.698: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:25:10 +0000 UTC (0+7 container statuses recorded) Nov 28 23:27:25.698: INFO: Container csi-attacher ready: false, restart count 0 Nov 28 23:27:25.698: INFO: Container csi-provisioner ready: false, restart count 0 Nov 28 23:27:25.698: INFO: Container csi-resizer ready: false, restart count 0 Nov 28 23:27:25.698: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 28 23:27:25.698: INFO: Container hostpath ready: false, restart count 0 Nov 28 23:27:25.698: INFO: Container liveness-probe ready: false, restart count 0 Nov 28 23:27:25.698: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 28 23:27:25.698: INFO: hostpath-symlink-prep-provisioning-5964 started at 2022-11-28 23:26:41 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container init-volume-provisioning-5964 ready: false, restart count 0 Nov 28 23:27:25.698: INFO: pod-265febdc-bbd3-45b9-93ae-1749df7c2ac4 started at 2022-11-28 23:27:03 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container write-pod ready: false, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-d9wpq started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-qxlqw started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.698: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-gqg8h started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:25.698: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:27:25.970: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:27:25.970: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:27:26.036: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 8957 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:27:10 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:27:10 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:27:10 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:27:10 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:27:26.036: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:27:26.105: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:27:26.175: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.175: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:27:26.175: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:27:26.175: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:27:26.175: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:27:26.175: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.175: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:27:26.175: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:27:26.175: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:27:26.175: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:26.175: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:27:26.176: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.176: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:27:26.176: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.176: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:27:26.176: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:27:26.176: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:27:26.176: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:27:26.176: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:27:26.176: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:27:26.176: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:27:26.176: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.176: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:27:26.176: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.176: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:27:26.176: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:27:26.176: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:27:26.176: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:27:26.176: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:27:26.176: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:27:26.456: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:27:26.456: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-8707" for this suite.
Find pod-99828920-35f1-4eb5-894d-226e6e6b50cf mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sshould\sresize\svolume\swhen\sPVC\sis\sedited\swhile\spod\sis\susing\sit$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 28 23:39:31.372: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc002aac7e0>: { s: "pod \"pod-2b009baf-6011-403a-8e47-7727e3df55c4\" is not Running: timed out waiting for the condition", } pod "pod-2b009baf-6011-403a-8e47-7727e3df55c4" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268from junit_12.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:34:30.594: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should resize volume when PVC is edited while pod is using it /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246 Nov 28 23:34:30.906: INFO: Creating resource for dynamic PV Nov 28 23:34:30.906: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-681-e2e-scsbc9d �[1mSTEP�[0m: creating a claim Nov 28 23:34:30.970: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Nov 28 23:39:31.372: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc002aac7e0>: { s: "pod \"pod-2b009baf-6011-403a-8e47-7727e3df55c4\" is not Running: timed out waiting for the condition", } pod "pod-2b009baf-6011-403a-8e47-7727e3df55c4" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:268 +0x3d0 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0002a7380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc0002a7380) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc0002a7380, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 28 23:39:31.373: INFO: Deleting pod "pod-2b009baf-6011-403a-8e47-7727e3df55c4" in namespace "volume-expand-681" Nov 28 23:39:31.446: INFO: Wait up to 5m0s for pod "pod-2b009baf-6011-403a-8e47-7727e3df55c4" to be fully deleted �[1mSTEP�[0m: Deleting pod Nov 28 23:39:31.509: INFO: Deleting pod "pod-2b009baf-6011-403a-8e47-7727e3df55c4" in namespace "volume-expand-681" �[1mSTEP�[0m: Deleting pvc Nov 28 23:39:31.634: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comt4n5j" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-681". �[1mSTEP�[0m: Found 5 events. Nov 28 23:39:31.830: INFO: At 2022-11-28 23:34:31 +0000 UTC - event for ebs.csi.aws.comt4n5j: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:39:31.830: INFO: At 2022-11-28 23:34:31 +0000 UTC - event for ebs.csi.aws.comt4n5j: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:39:31.830: INFO: At 2022-11-28 23:34:31 +0000 UTC - event for ebs.csi.aws.comt4n5j: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "volume-expand-681/ebs.csi.aws.comt4n5j" Nov 28 23:39:31.830: INFO: At 2022-11-28 23:34:41 +0000 UTC - event for ebs.csi.aws.comt4n5j: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-681-e2e-scsbc9d": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:39:31.830: INFO: At 2022-11-28 23:36:02 +0000 UTC - event for ebs.csi.aws.comt4n5j: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "volume-expand-681-e2e-scsbc9d": rpc error: code = Internal desc = Could not create volume "pvc-e743ebd0-1230-44dc-af5b-2d3bd1c38441": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:39:31.892: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:39:31.893: INFO: Nov 28 23:39:31.956: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:32.025: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 25977 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:38:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:39:02 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:32.026: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:32.191: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:32.267: INFO: busybox-scheduling-5968efa6-3643-4b09-9605-4298963b5725 started at 2022-11-28 23:39:12 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container busybox-scheduling-5968efa6-3643-4b09-9605-4298963b5725 ready: true, restart count 0 Nov 28 23:39:32.267: INFO: externalname-service-2x5rc started at 2022-11-28 23:39:28 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container externalname-service ready: true, restart count 0 Nov 28 23:39:32.267: INFO: annotationupdate58e6a497-5ca8-4c43-a391-4c94b518b80f started at 2022-11-28 23:39:23 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container client-container ready: true, restart count 0 Nov 28 23:39:32.267: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container coredns ready: true, restart count 0 Nov 28 23:39:32.267: INFO: pod-subpath-test-secret-ctrd started at 2022-11-28 23:39:07 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container test-container-subpath-secret-ctrd ready: false, restart count 0 Nov 28 23:39:32.267: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:32.267: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:32.267: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:32.267: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:32.267: INFO: pod-adoption-release started at 2022-11-28 23:39:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container pod-adoption-release ready: false, restart count 0 Nov 28 23:39:32.267: INFO: startup-19549d57-ef5e-4e10-901b-65ce5445ea92 started at 2022-11-28 23:39:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container busybox ready: false, restart count 0 Nov 28 23:39:32.267: INFO: my-hostname-basic-85e084c4-0ff0-4741-a3ff-25529682c210-2dtxg started at 2022-11-28 23:39:23 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container my-hostname-basic-85e084c4-0ff0-4741-a3ff-25529682c210 ready: true, restart count 0 Nov 28 23:39:32.267: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:32.267: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:32.267: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.267: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:39:32.563: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:39:32.563: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:32.626: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 26005 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8286":"csi-mock-csi-mock-volumes-8286","ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:35:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:39:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311,DevicePath:,},},Config:nil,},} Nov 28 23:39:32.627: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:32.694: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:32.761: INFO: webhook-to-be-mutated started at 2022-11-28 23:38:29 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Init container webhook-added-init-container ready: false, restart count 0 Nov 28 23:39:32.761: INFO: Container example ready: false, restart count 0 Nov 28 23:39:32.761: INFO: csi-mockplugin-0 started at 2022-11-28 23:37:31 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:32.761: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container mock ready: true, restart count 0 Nov 28 23:39:32.761: INFO: pod-subpath-test-inlinevolume-r9rj started at 2022-11-28 23:39:15 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Init container init-volume-inlinevolume-r9rj ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container test-container-subpath-inlinevolume-r9rj ready: true, restart count 0 Nov 28 23:39:32.761: INFO: hostexec-ip-172-20-43-89.us-west-2.compute.internal-jpffs started at 2022-11-28 23:39:17 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:39:32.761: INFO: startup-script started at 2022-11-28 23:38:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Container startup-script ready: true, restart count 0 Nov 28 23:39:32.761: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:37:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:39:32.761: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:32.761: INFO: pod-server-1 started at 2022-11-28 23:39:04 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Init container init ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:39:32.761: INFO: exec-volume-test-preprovisionedpv-v2kj started at 2022-11-28 23:39:28 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Container exec-container-preprovisionedpv-v2kj ready: false, restart count 0 Nov 28 23:39:32.761: INFO: pvc-volume-tester-hj2f9 started at 2022-11-28 23:37:41 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:32.761: INFO: Container volume-tester ready: true, restart count 0 Nov 28 23:39:32.761: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:32.761: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:32.761: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:33.071: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:39:33.071: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:33.134: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 23138 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:34:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:33.134: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:33.200: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:33.270: INFO: inline-volume-vhkw9 started at 2022-11-28 23:37:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.270: INFO: Container volume-tester ready: false, restart count 0 Nov 28 23:39:33.270: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:33.270: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:33.270: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:33.270: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:33.270: INFO: execpod74chz started at 2022-11-28 23:39:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.270: INFO: Container agnhost-container ready: false, restart count 0 Nov 28 23:39:33.270: INFO: busybox-0d1169e6-d555-4838-9ecb-a54946f6911e started at 2022-11-28 23:38:54 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.270: INFO: Container busybox ready: true, restart count 0 Nov 28 23:39:33.270: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:33.270: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:33.270: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:33.270: INFO: pod-client started at 2022-11-28 23:39:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.270: INFO: Container pod-client ready: true, restart count 0 Nov 28 23:39:33.270: INFO: deployment-shared-map-item-removal-55649fd747-9v85c started at 2022-11-28 23:38:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.270: INFO: Container nginx ready: false, restart count 0 Nov 28 23:39:33.515: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:39:33.515: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:33.596: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 26862 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:39:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:39:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:33.596: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:33.661: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:33.731: INFO: pod-size-memory-volume-be44bc97-85a4-4ff6-a3a2-2badfb1baa0b started at 2022-11-28 23:39:27 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.731: INFO: Container busybox-main-container ready: true, restart count 0 Nov 28 23:39:33.731: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.731: INFO: Container coredns ready: true, restart count 0 Nov 28 23:39:33.731: INFO: deployment-shared-map-item-removal-55649fd747-bgf96 started at 2022-11-28 23:38:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.731: INFO: Container nginx ready: false, restart count 0 Nov 28 23:39:33.731: INFO: labelsupdatec1328a21-c527-4e3f-8d1c-1f0896e76e55 started at 2022-11-28 23:39:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.731: INFO: Container client-container ready: true, restart count 0 Nov 28 23:39:33.731: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:33.731: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:33.731: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:33.731: INFO: csi-mockplugin-0 started at 2022-11-28 23:38:33 +0000 UTC (0+4 container statuses recorded) Nov 28 23:39:33.731: INFO: Container busybox ready: true, restart count 0 Nov 28 23:39:33.731: INFO: Container csi-provisioner ready: false, restart count 1 Nov 28 23:39:33.731: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:39:33.731: INFO: Container mock ready: true, restart count 0 Nov 28 23:39:33.731: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:33.731: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:33.731: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:33.731: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:33.731: INFO: externalname-service-9s7rz started at 2022-11-28 23:39:28 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:33.731: INFO: Container externalname-service ready: true, restart count 0 Nov 28 23:39:34.101: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:39:34.101: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:34.217: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 22365 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:39:34.218: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:34.309: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:34.383: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:39:34.383: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:39:34.383: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:39:34.383: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:39:34.383: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:39:34.383: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:39:34.383: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:34.383: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:39:34.383: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:39:34.383: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:39:34.383: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:39:34.383: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:39:34.383: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:39:34.383: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:39:34.383: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:39:34.383: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:39:34.666: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:39:34.666: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-681" for this suite.
Find pod-2b009baf-6011-403a-8e47-7727e3df55c4 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:45:56.856: Unexpected error: <*errors.errorString | 0xc00306d060>: { s: "pod \"pod-ecab8f56-6b16-431c-8caa-ef1e3d793026\" is not Running: timed out waiting for the condition", } pod "pod-ecab8f56-6b16-431c-8caa-ef1e3d793026" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_03.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:40:55.702: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:40:56.048: INFO: Creating resource for dynamic PV Nov 28 23:40:56.048: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-5609-e2e-scd75zf �[1mSTEP�[0m: creating a claim Nov 28 23:40:56.158: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-5609 with fsgroup 1000 Nov 28 23:45:56.856: FAIL: Unexpected error: <*errors.errorString | 0xc00306d060>: { s: "pod \"pod-ecab8f56-6b16-431c-8caa-ef1e3d793026\" is not Running: timed out waiting for the condition", } pod "pod-ecab8f56-6b16-431c-8caa-ef1e3d793026" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc002f15b80, 0xc00205d0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000787800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000787800) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000787800, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 28 23:45:56.923: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comghh72" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-5609". �[1mSTEP�[0m: Found 5 events. Nov 28 23:45:57.124: INFO: At 2022-11-28 23:40:56 +0000 UTC - event for ebs.csi.aws.comghh72: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:45:57.124: INFO: At 2022-11-28 23:40:56 +0000 UTC - event for ebs.csi.aws.comghh72: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:45:57.124: INFO: At 2022-11-28 23:40:56 +0000 UTC - event for ebs.csi.aws.comghh72: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-5609/ebs.csi.aws.comghh72" Nov 28 23:45:57.124: INFO: At 2022-11-28 23:41:06 +0000 UTC - event for ebs.csi.aws.comghh72: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5609-e2e-scd75zf": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:45:57.124: INFO: At 2022-11-28 23:41:17 +0000 UTC - event for ebs.csi.aws.comghh72: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5609-e2e-scd75zf": rpc error: code = Internal desc = Could not create volume "pvc-3cd8e230-3c56-439a-a457-09b1b2d4052e": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:45:57.190: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:45:57.190: INFO: pod-ecab8f56-6b16-431c-8caa-ef1e3d793026 Pending [] Nov 28 23:45:57.190: INFO: Nov 28 23:45:57.257: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:45:57.323: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 40707 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-956":"csi-mock-csi-mock-volumes-956","ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:45:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-11-28 23:45:55 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:45:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:45:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:45:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:45:43 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-956^4,DevicePath:,},},Config:nil,},} Nov 28 23:45:57.324: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:45:57.392: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:45:57.468: INFO: test-rolling-update-with-lb-7db9cf5ff8-bjp4c started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.468: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:45:57.468: INFO: csi-mockplugin-0 started at 2022-11-28 23:45:49 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:57.468: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:45:57.468: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:45:57.469: INFO: Container mock ready: true, restart count 0 Nov 28 23:45:57.469: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:45:57.469: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:45:57.469: INFO: csi-mockplugin-0 started at 2022-11-28 23:45:18 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:57.469: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:45:57.469: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:45:57.469: INFO: Container mock ready: true, restart count 0 Nov 28 23:45:57.469: INFO: affinity-nodeport-timeout-4865j started at 2022-11-28 23:45:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Nov 28 23:45:57.469: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:45:57.469: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:45:18 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:45:57.469: INFO: concurrent-27827985-x2jqx started at 2022-11-28 23:45:00 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container c ready: false, restart count 0 Nov 28 23:45:57.469: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:45:49 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:45:57.469: INFO: pvc-volume-tester-d4d5w started at 2022-11-28 23:45:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container volume-tester ready: false, restart count 0 Nov 28 23:45:57.469: INFO: replace-27827985-95blh started at 2022-11-28 23:45:00 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container c ready: true, restart count 0 Nov 28 23:45:57.469: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:57.469: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:45:57.469: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:45:57.469: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:45:57.469: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.469: INFO: Container coredns ready: true, restart count 0 Nov 28 23:45:57.765: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:45:57.765: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:45:57.832: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 38659 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:45:57.833: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:45:57.901: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:45:57.971: INFO: pod-secrets-654a65b9-5b94-43dc-a57e-b94afc970505 started at 2022-11-28 23:45:07 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:57.972: INFO: Container creates-volume-test ready: true, restart count 0 Nov 28 23:45:57.972: INFO: Container dels-volume-test ready: true, restart count 0 Nov 28 23:45:57.972: INFO: Container upds-volume-test ready: true, restart count 0 Nov 28 23:45:57.972: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:57.972: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:45:57.972: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:45:57.972: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:45:57.972: INFO: test-rolling-update-with-lb-7db9cf5ff8-fd9gb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.972: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:45:57.972: INFO: probe-test-4d88581e-365e-4eb9-b0e9-86c02db731ca started at 2022-11-28 23:45:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.972: INFO: Container probe-test-4d88581e-365e-4eb9-b0e9-86c02db731ca ready: false, restart count 0 Nov 28 23:45:57.972: INFO: run-test-3 started at 2022-11-28 23:45:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:57.972: INFO: Container run-test-3 ready: true, restart count 0 Nov 28 23:45:57.972: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:45:57.972: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:45:57.972: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:45:58.244: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:45:58.245: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:45:58.311: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 39281 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:45:58.312: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:45:58.380: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:45:58.456: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:45:58.456: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:45:58.456: INFO: liveness-98271573-f7d8-4da0-a6a6-7d2ee626de6f started at 2022-11-28 23:44:43 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:45:58.456: INFO: probe-test-a0145842-1948-4fec-bc90-802e8294bfe9 started at 2022-11-28 23:45:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Container probe-test-a0145842-1948-4fec-bc90-802e8294bfe9 ready: false, restart count 0 Nov 28 23:45:58.456: INFO: pod-subpath-test-preprovisionedpv-9fzs started at 2022-11-28 23:45:58 +0000 UTC (2+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Init container init-volume-preprovisionedpv-9fzs ready: false, restart count 0 Nov 28 23:45:58.456: INFO: Init container test-init-volume-preprovisionedpv-9fzs ready: false, restart count 0 Nov 28 23:45:58.456: INFO: Container test-container-subpath-preprovisionedpv-9fzs ready: false, restart count 0 Nov 28 23:45:58.456: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:58.456: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:45:58.456: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:45:58.456: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:45:58.456: INFO: affinity-nodeport-timeout-khg44 started at 2022-11-28 23:45:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Nov 28 23:45:58.456: INFO: execpod-affinitycmqhj started at 2022-11-28 23:45:12 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:45:58.456: INFO: hostexec-ip-172-20-51-62.us-west-2.compute.internal-6z85q started at 2022-11-28 23:45:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.456: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:45:58.788: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:45:58.788: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:45:58.854: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 37644 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:41:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:43:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f,DevicePath:,},},Config:nil,},} Nov 28 23:45:58.855: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:45:58.923: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:45:58.992: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.992: INFO: Container coredns ready: true, restart count 0 Nov 28 23:45:58.992: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:45:58.992: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:45:58.992: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:45:58.993: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:58.993: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:45:58.993: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:45:58.993: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:45:58.993: INFO: test-rolling-update-with-lb-7db9cf5ff8-s9xpb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.993: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:45:58.993: INFO: affinity-nodeport-timeout-75plv started at 2022-11-28 23:45:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:58.993: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Nov 28 23:45:59.267: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:45:59.267: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:45:59.334: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 34762 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:42:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:45:59.334: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:45:59.403: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:45:59.474: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.474: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:45:59.474: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.474: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:45:59.474: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:45:59.474: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:45:59.474: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:45:59.474: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.474: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:45:59.474: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:45:59.474: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:45:59.474: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:45:59.475: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:45:59.475: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:45:59.475: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:45:59.475: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.475: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:45:59.475: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.475: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:45:59.475: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:45:59.475: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:45:59.475: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:45:59.475: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:45:59.475: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.475: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:45:59.475: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:45:59.475: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:45:59.475: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:45:59.475: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:45:59.475: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:45:59.753: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:45:59.753: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-5609" for this suite.
Find pod-ecab8f56-6b16-431c-8caa-ef1e3d793026 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:38:09.416: Unexpected error: <*errors.errorString | 0xc002fc8440>: { s: "pod \"pod-87287f4b-80ed-4b8c-bcfc-90d9db6d49dd\" is not Running: timed out waiting for the condition", } pod "pod-87287f4b-80ed-4b8c-bcfc-90d9db6d49dd" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_07.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:33:08.647: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:33:08.964: INFO: Creating resource for dynamic PV Nov 28 23:33:08.964: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-977-e2e-sc59ssf �[1mSTEP�[0m: creating a claim Nov 28 23:33:09.029: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-977 with fsgroup 1000 Nov 28 23:38:09.415: FAIL: Unexpected error: <*errors.errorString | 0xc002fc8440>: { s: "pod \"pod-87287f4b-80ed-4b8c-bcfc-90d9db6d49dd\" is not Running: timed out waiting for the condition", } pod "pod-87287f4b-80ed-4b8c-bcfc-90d9db6d49dd" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc00369b8c0, 0xc0024330d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00046fc80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc00046fc80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc00046fc80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 28 23:38:09.480: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com5wfrt" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-977". �[1mSTEP�[0m: Found 4 events. Nov 28 23:38:09.677: INFO: At 2022-11-28 23:33:09 +0000 UTC - event for ebs.csi.aws.com5wfrt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:38:09.677: INFO: At 2022-11-28 23:33:09 +0000 UTC - event for ebs.csi.aws.com5wfrt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:38:09.677: INFO: At 2022-11-28 23:33:09 +0000 UTC - event for ebs.csi.aws.com5wfrt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-977/ebs.csi.aws.com5wfrt" Nov 28 23:38:09.677: INFO: At 2022-11-28 23:33:19 +0000 UTC - event for ebs.csi.aws.com5wfrt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-977-e2e-sc59ssf": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:38:09.741: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:38:09.741: INFO: pod-87287f4b-80ed-4b8c-bcfc-90d9db6d49dd Pending [] Nov 28 23:38:09.741: INFO: Nov 28 23:38:09.808: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:38:09.872: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 23369 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-762":"ip-172-20-33-9.us-west-2.compute.internal","ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:37:25 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:37:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:32 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:32 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:32 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:32 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-762^9c426dc2-6f75-11ed-8c41-729b4c0c1434],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-762^9c426dc2-6f75-11ed-8c41-729b4c0c1434,DevicePath:,},},Config:nil,},} Nov 28 23:38:09.872: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:38:09.937: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:38:10.006: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:37:21 +0000 UTC (0+7 container statuses recorded) Nov 28 23:38:10.006: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:38:10.006: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:10.006: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:38:10.006: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.006: INFO: Container coredns ready: true, restart count 0 Nov 28 23:38:10.006: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-276ks started at 2022-11-28 23:37:56 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.006: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:38:10.006: INFO: liveness-6b37a8a0-cfe5-43bc-bd38-18a946016b89 started at 2022-11-28 23:35:48 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.006: INFO: Container agnhost-container ready: false, restart count 4 Nov 28 23:38:10.006: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:38:10.006: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:38:10.006: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:38:10.006: INFO: deployment-585449566-92mcd started at 2022-11-28 23:36:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.006: INFO: Container nginx ready: false, restart count 0 Nov 28 23:38:10.006: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.006: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:38:10.308: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:38:10.308: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:38:10.372: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 24097 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8286":"csi-mock-csi-mock-volumes-8286","ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:35:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311,DevicePath:,},},Config:nil,},} Nov 28 23:38:10.373: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:38:10.438: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:38:10.515: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:38:10.515: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:38:10.515: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:38:10.515: INFO: pvc-volume-tester-hj2f9 started at 2022-11-28 23:37:41 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.515: INFO: Container volume-tester ready: true, restart count 0 Nov 28 23:38:10.515: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:10.515: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:38:10.515: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:10.515: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:38:10.515: INFO: csi-mockplugin-0 started at 2022-11-28 23:37:31 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:10.515: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:38:10.515: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:38:10.515: INFO: Container mock ready: true, restart count 0 Nov 28 23:38:10.515: INFO: hostexec-ip-172-20-43-89.us-west-2.compute.internal-fd8bz started at 2022-11-28 23:37:59 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.515: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:38:10.515: INFO: startup-script started at 2022-11-28 23:38:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.515: INFO: Container startup-script ready: false, restart count 0 Nov 28 23:38:10.515: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:37:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:10.515: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:38:10.817: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:38:10.817: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:38:10.881: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 23138 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:34:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:38:10.881: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:38:10.950: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:38:11.019: INFO: inline-volume-vhkw9 started at 2022-11-28 23:37:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.019: INFO: Container volume-tester ready: false, restart count 0 Nov 28 23:38:11.019: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:11.019: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:38:11.019: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:11.019: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:38:11.019: INFO: liveness-748a8281-79e6-49cf-a8f4-ab53a92221e4 started at 2022-11-28 23:35:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.019: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:38:11.019: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:38:11.019: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:38:11.019: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:38:11.019: INFO: metadata-volume-9bb88397-aba6-4d11-af71-e2679a863814 started at 2022-11-28 23:38:04 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.019: INFO: Container client-container ready: false, restart count 0 Nov 28 23:38:11.019: INFO: pod-subpath-test-inlinevolume-r7kf started at 2022-11-28 23:38:06 +0000 UTC (1+2 container statuses recorded) Nov 28 23:38:11.019: INFO: Init container test-init-subpath-inlinevolume-r7kf ready: true, restart count 0 Nov 28 23:38:11.019: INFO: Container test-container-subpath-inlinevolume-r7kf ready: false, restart count 0 Nov 28 23:38:11.019: INFO: Container test-container-volume-inlinevolume-r7kf ready: false, restart count 0 Nov 28 23:38:11.274: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:38:11.274: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:38:11.338: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 23931 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9568":"csi-mock-csi-mock-volumes-9568","ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:38:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:38:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:38:11.338: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:38:11.405: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:38:11.472: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:38:11.472: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:38:11.472: INFO: deployment-585449566-crzxz started at 2022-11-28 23:36:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Container nginx ready: false, restart count 0 Nov 28 23:38:11.472: INFO: csi-mockplugin-0 started at 2022-11-28 23:37:35 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:11.472: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:38:11.472: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:38:11.472: INFO: Container mock ready: true, restart count 0 Nov 28 23:38:11.472: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:11.472: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:38:11.472: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:11.472: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:38:11.472: INFO: deployment-55649fd747-bw7j9 started at 2022-11-28 23:36:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Container nginx ready: false, restart count 0 Nov 28 23:38:11.472: INFO: csi-mockplugin-attacher-0 started at 2022-11-28 23:37:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:38:11.472: INFO: boom-server started at 2022-11-28 23:38:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Container boom-server ready: true, restart count 0 Nov 28 23:38:11.472: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Container coredns ready: true, restart count 0 Nov 28 23:38:11.472: INFO: hostexec-ip-172-20-59-24.us-west-2.compute.internal-4gq8d started at 2022-11-28 23:37:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.472: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:38:11.752: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:38:11.752: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:38:11.815: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 22365 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:37:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:38:11.816: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:38:11.882: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:38:11.958: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:38:11.958: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:38:11.958: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:38:11.958: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:11.958: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:38:11.958: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:38:11.958: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:38:11.958: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:38:11.958: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:38:11.958: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:38:11.958: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:38:11.958: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:38:11.958: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:38:11.958: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:38:11.958: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:38:11.958: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:38:12.208: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:38:12.208: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-977" for this suite.
Find pod-87287f4b-80ed-4b8c-bcfc-90d9db6d49dd mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:25:04.408: Unexpected error: <*errors.errorString | 0xc002f9f0c0>: { s: "pod \"pod-32f4b4a9-0e5d-4f54-9c72-b923385d3515\" is not Running: timed out waiting for the condition", } pod "pod-32f4b4a9-0e5d-4f54-9c72-b923385d3515" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_13.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:20:03.661: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:20:03.967: INFO: Creating resource for dynamic PV Nov 28 23:20:03.967: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-1720-e2e-scnc6jm �[1mSTEP�[0m: creating a claim Nov 28 23:20:04.031: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-1720 with fsgroup 1000 Nov 28 23:25:04.408: FAIL: Unexpected error: <*errors.errorString | 0xc002f9f0c0>: { s: "pod \"pod-32f4b4a9-0e5d-4f54-9c72-b923385d3515\" is not Running: timed out waiting for the condition", } pod "pod-32f4b4a9-0e5d-4f54-9c72-b923385d3515" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc002126840, 0xc000a950d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000a37c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000a37c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000a37c80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 28 23:25:04.470: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.commmq65" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-1720". �[1mSTEP�[0m: Found 5 events. Nov 28 23:25:04.657: INFO: At 2022-11-28 23:20:04 +0000 UTC - event for ebs.csi.aws.commmq65: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:25:04.657: INFO: At 2022-11-28 23:20:04 +0000 UTC - event for ebs.csi.aws.commmq65: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:25:04.657: INFO: At 2022-11-28 23:20:04 +0000 UTC - event for ebs.csi.aws.commmq65: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-1720/ebs.csi.aws.commmq65" Nov 28 23:25:04.657: INFO: At 2022-11-28 23:20:14 +0000 UTC - event for ebs.csi.aws.commmq65: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-1720-e2e-scnc6jm": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:25:04.657: INFO: At 2022-11-28 23:20:25 +0000 UTC - event for ebs.csi.aws.commmq65: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-1720-e2e-scnc6jm": rpc error: code = Internal desc = Could not create volume "pvc-85efe2df-3928-4d9b-8f63-b5c22123aa59": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:25:04.719: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:25:04.719: INFO: pod-32f4b4a9-0e5d-4f54-9c72-b923385d3515 Pending [] Nov 28 23:25:04.719: INFO: Nov 28 23:25:04.782: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:25:04.843: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 7034 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-4714":"ip-172-20-33-9.us-west-2.compute.internal","csi-mock-csi-mock-volumes-8632":"csi-mock-csi-mock-volumes-8632","ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:20:28 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:20:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:21:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:21:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:21:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:21:01 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-4714^3da39203-6f73-11ed-9ff4-dac3ce5938e2],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-4714^3da39203-6f73-11ed-9ff4-dac3ce5938e2,DevicePath:,},},Config:nil,},} Nov 28 23:25:04.844: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:25:04.908: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:25:05.042: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:25:05.042: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:25:05.042: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:25:05.042: INFO: pod-subpath-test-preprovisionedpv-4ldf started at 2022-11-28 23:22:43 +0000 UTC (2+1 container statuses recorded) Nov 28 23:25:05.042: INFO: Init container init-volume-preprovisionedpv-4ldf ready: false, restart count 0 Nov 28 23:25:05.042: INFO: Init container test-init-subpath-preprovisionedpv-4ldf ready: false, restart count 0 Nov 28 23:25:05.042: INFO: Container test-container-subpath-preprovisionedpv-4ldf ready: false, restart count 0 Nov 28 23:25:05.042: INFO: pod-4b13b367-9436-447a-9389-ba41e0e58746 started at 2022-11-28 23:20:44 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.042: INFO: Container write-pod ready: false, restart count 0 Nov 28 23:25:05.042: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-jrjjf started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.042: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-dd5fh started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.043: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-6t57g started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:25:05.043: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:25:05.043: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:22:40 +0000 UTC (0+7 container statuses recorded) Nov 28 23:25:05.043: INFO: Container csi-attacher ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container csi-provisioner ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container csi-resizer ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container csi-snapshotter ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container hostpath ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container liveness-probe ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container node-driver-registrar ready: false, restart count 0 Nov 28 23:25:05.043: INFO: pod-server-2 started at 2022-11-28 23:21:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container agnhost-container ready: false, restart count 0 Nov 28 23:25:05.043: INFO: pod-subpath-test-preprovisionedpv-pml4 started at 2022-11-28 23:22:29 +0000 UTC (1+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Init container init-volume-preprovisionedpv-pml4 ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container test-container-subpath-preprovisionedpv-pml4 ready: false, restart count 0 Nov 28 23:25:05.043: INFO: pod-subpath-test-inlinevolume-hn7q started at 2022-11-28 23:23:19 +0000 UTC (2+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Init container init-volume-inlinevolume-hn7q ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Init container test-init-volume-inlinevolume-hn7q ready: false, restart count 0 Nov 28 23:25:05.043: INFO: Container test-container-subpath-inlinevolume-hn7q ready: false, restart count 0 Nov 28 23:25:05.043: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:20:00 +0000 UTC (0+7 container statuses recorded) Nov 28 23:25:05.043: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:05.043: INFO: pvc-volume-tester-hr5jb started at 2022-11-28 23:21:04 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container volume-tester ready: false, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-dstxj started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.043: INFO: pod-server-1 started at 2022-11-28 23:20:48 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-drjbg started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-r9k29 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.043: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:25:05.043: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:05.043: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container coredns ready: true, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-qgkzt started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-tgb48 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-sbskj started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-277wv started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:25:05.043: INFO: csi-mockplugin-0 started at 2022-11-28 23:19:54 +0000 UTC (0+3 container statuses recorded) Nov 28 23:25:05.043: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container driver-registrar ready: true, restart count 0 Nov 28 23:25:05.043: INFO: Container mock ready: true, restart count 0 Nov 28 23:25:05.043: INFO: pod-subpath-test-preprovisionedpv-w7dp started at 2022-11-28 23:22:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container test-container-subpath-preprovisionedpv-w7dp ready: false, restart count 0 Nov 28 23:25:05.043: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-pv6c2 started at 2022-11-28 23:22:15 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:25:05.043: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-h9dj2 started at 2022-11-28 23:22:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:25:05.043: INFO: netserver-0 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container webserver ready: false, restart count 0 Nov 28 23:25:05.043: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-z68b9 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.043: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.467: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:25:05.467: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:25:05.529: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 6327 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-28 23:22:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:22:03 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:25:05.530: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:25:05.593: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:25:05.660: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:25:05.660: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-xtzkx started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: busybox-df981fcf-34dd-4d33-972f-8fbe9a730ecc started at 2022-11-28 23:24:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container busybox ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-bxd52 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: pod-service-account-2eeae25a-9046-4e82-9198-b23c568eb5de started at 2022-11-28 23:22:10 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container test ready: false, restart count 0 Nov 28 23:25:05.660: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:25:05.660: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:25:05.660: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:05.660: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-sdflf started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-flrbd started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: busybox-user-65534-625e0c38-ec9d-4a82-9e0c-b7f9ab373147 started at 2022-11-28 23:23:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container busybox-user-65534-625e0c38-ec9d-4a82-9e0c-b7f9ab373147 ready: false, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-d9z4t started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: test-pod-3 started at 2022-11-28 23:22:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container token-test ready: true, restart count 0 Nov 28 23:25:05.660: INFO: termination-message-container5295c1ca-0d54-4b59-bfa1-69951bdc3eab started at 2022-11-28 23:24:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container termination-message-container ready: false, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-rw6kp started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-kzswp started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-p6vgv started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: false, restart count 0 Nov 28 23:25:05.660: INFO: privileged-pod started at 2022-11-28 23:24:43 +0000 UTC (0+2 container statuses recorded) Nov 28 23:25:05.660: INFO: Container not-privileged-container ready: false, restart count 0 Nov 28 23:25:05.660: INFO: Container privileged-container ready: false, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-j5xkz started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-qvrrq started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:05.660: INFO: netserver-1 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:05.660: INFO: Container webserver ready: true, restart count 0 Nov 28 23:25:06.018: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:25:06.018: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:25:06.080: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 7270 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-11-28 23:22:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:25:06.081: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:25:06.145: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-hd7zs started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: test-pod-2 started at 2022-11-28 23:22:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container token-test ready: false, restart count 0 Nov 28 23:25:06.211: INFO: test-pod-1 started at 2022-11-28 23:22:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container token-test ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-xh7kh started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-4vt6f started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-dzzdn started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: logs-generator started at 2022-11-28 23:24:32 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container logs-generator ready: false, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-q9dwk started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-zrxm7 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:25:06.211: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:25:06.211: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:06.211: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:25:06.211: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:25:06.211: INFO: pod-client started at 2022-11-28 23:20:37 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container pod-client ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-9w68m started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-glk2d started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: terminate-cmd-rpa60ff7315-049a-4913-b7bd-efdd8f22fdac started at 2022-11-28 23:23:41 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container terminate-cmd-rpa ready: false, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-s68jn started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-8kqzt started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.211: INFO: netserver-2 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.211: INFO: Container webserver ready: false, restart count 0 Nov 28 23:25:06.490: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:25:06.490: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:25:06.552: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 6851 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-9863":"ip-172-20-59-24.us-west-2.compute.internal","ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:20:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:22:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:22:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:22:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-9863^783448ea-6f73-11ed-a5aa-b67a39298b1d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-9863^783448ea-6f73-11ed-a5aa-b67a39298b1d,DevicePath:,},},Config:nil,},} Nov 28 23:25:06.552: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:25:06.616: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-tlf72 started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:25:06.685: INFO: hostpath-injector started at 2022-11-28 23:22:11 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container hostpath-injector ready: false, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-sp6hj started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: netserver-3 started at 2022-11-28 23:23:26 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container webserver ready: false, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-jvz2t started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-747bf started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-rs47m started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:25:06.685: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-7gg2s started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:21:44 +0000 UTC (0+7 container statuses recorded) Nov 28 23:25:06.685: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:06.685: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-khs7v started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container coredns ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-d9wpq started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-qxlqw started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.685: INFO: cleanup40-97dc542c-f633-4b68-80cf-71bbab781808-gqg8h started at 2022-11-28 23:20:50 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:06.685: INFO: Container cleanup40-97dc542c-f633-4b68-80cf-71bbab781808 ready: true, restart count 0 Nov 28 23:25:06.964: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:25:06.964: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:25:07.025: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 6362 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:10 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:10 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:22:10 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:22:10 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:25:07.026: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:25:07.090: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:25:07.157: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:25:07.157: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:25:07.157: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:25:07.157: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:25:07.157: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:25:07.157: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:25:07.157: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:25:07.157: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:25:07.157: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:25:07.157: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:25:07.157: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:25:07.157: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:25:07.157: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:25:07.157: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:25:07.157: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:25:07.157: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:25:07.426: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:25:07.426: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-1720" for this suite.
Find pod-32f4b4a9-0e5d-4f54-9c72-b923385d3515 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:49:16.364: Unexpected error: <*errors.errorString | 0xc0052008d0>: { s: "pod \"pod-e4f12b0b-f4ea-4f1a-a91c-50d400605cef\" is not Running: timed out waiting for the condition", } pod "pod-e4f12b0b-f4ea-4f1a-a91c-50d400605cef" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_06.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:44:15.560: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:44:15.895: INFO: Creating resource for dynamic PV Nov 28 23:44:15.895: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-5643-e2e-sclrwqr �[1mSTEP�[0m: creating a claim Nov 28 23:44:15.964: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-5643 with fsgroup 1000 Nov 28 23:49:16.364: FAIL: Unexpected error: <*errors.errorString | 0xc0052008d0>: { s: "pod \"pod-e4f12b0b-f4ea-4f1a-a91c-50d400605cef\" is not Running: timed out waiting for the condition", } pod "pod-e4f12b0b-f4ea-4f1a-a91c-50d400605cef" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc003284f20, 0xc0018430d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000d53680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000d53680) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000d53680, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 28 23:49:16.435: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comvxvtt" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-5643". �[1mSTEP�[0m: Found 5 events. Nov 28 23:49:16.639: INFO: At 2022-11-28 23:44:16 +0000 UTC - event for ebs.csi.aws.comvxvtt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:49:16.639: INFO: At 2022-11-28 23:44:16 +0000 UTC - event for ebs.csi.aws.comvxvtt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:49:16.639: INFO: At 2022-11-28 23:44:16 +0000 UTC - event for ebs.csi.aws.comvxvtt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-5643/ebs.csi.aws.comvxvtt" Nov 28 23:49:16.639: INFO: At 2022-11-28 23:44:26 +0000 UTC - event for ebs.csi.aws.comvxvtt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5643-e2e-sclrwqr": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:49:16.639: INFO: At 2022-11-28 23:44:37 +0000 UTC - event for ebs.csi.aws.comvxvtt: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-5643-e2e-sclrwqr": rpc error: code = Internal desc = Could not create volume "pvc-29aefb78-d6a6-4b94-aa35-c678a93addca": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:49:16.711: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:49:16.711: INFO: pod-e4f12b0b-f4ea-4f1a-a91c-50d400605cef Pending [] Nov 28 23:49:16.711: INFO: Nov 28 23:49:16.779: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:16.847: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 43718 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:48:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:49:16.847: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:16.916: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:16.990: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:49:16.990: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:49:16.990: INFO: pod-init-4cadbde1-9293-41bf-b984-f8f6a5d9dd5c started at 2022-11-28 23:49:09 +0000 UTC (2+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Init container init1 ready: false, restart count 1 Nov 28 23:49:16.990: INFO: Init container init2 ready: false, restart count 0 Nov 28 23:49:16.990: INFO: Container run1 ready: false, restart count 0 Nov 28 23:49:16.990: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:49:16.990: INFO: netserver-0 started at 2022-11-28 23:48:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Container webserver ready: true, restart count 0 Nov 28 23:49:16.990: INFO: suspend-true-to-false-sp2tx started at 2022-11-28 23:49:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Container c ready: false, restart count 0 Nov 28 23:49:16.990: INFO: suspend-true-to-false-z4sp8 started at 2022-11-28 23:49:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Container c ready: false, restart count 0 Nov 28 23:49:16.990: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:49:16.990: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:16.990: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:16.990: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:16.990: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:16.990: INFO: Container coredns ready: true, restart count 0 Nov 28 23:49:18.415: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:49:18.415: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:49:18.482: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 44966 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:49:18.483: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:49:18.552: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:49:18.621: INFO: pod-ephm-test-projected-hcq2 started at 2022-11-28 23:47:33 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:18.621: INFO: Container test-container-subpath-projected-hcq2 ready: false, restart count 0 Nov 28 23:49:18.621: INFO: test-container-pod started at 2022-11-28 23:49:18 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:18.621: INFO: Container webserver ready: false, restart count 0 Nov 28 23:49:18.621: INFO: netserver-1 started at 2022-11-28 23:48:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:18.621: INFO: Container webserver ready: true, restart count 0 Nov 28 23:49:18.621: INFO: host-test-container-pod started at 2022-11-28 23:49:18 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:18.621: INFO: Container agnhost-container ready: false, restart count 0 Nov 28 23:49:18.621: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:49:18.621: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:49:18.621: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:49:18.621: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:49:18.621: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:18.621: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:18.621: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:19.090: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:49:19.090: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:49:19.159: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 41438 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:49:19.160: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:49:19.232: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:49:19.302: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:49:19.302: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:19.302: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:19.302: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:19.302: INFO: slave started at 2022-11-28 23:48:54 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container cntr ready: true, restart count 0 Nov 28 23:49:19.302: INFO: default started at 2022-11-28 23:49:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container cntr ready: true, restart count 0 Nov 28 23:49:19.302: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:49:19.302: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:49:19.302: INFO: master started at 2022-11-28 23:48:49 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container cntr ready: true, restart count 0 Nov 28 23:49:19.302: INFO: webserver-pod started at 2022-11-28 23:46:39 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container agnhost ready: false, restart count 0 Nov 28 23:49:19.302: INFO: test-webserver-5baf5b03-7bc7-4db6-b317-09e6eb20d71d started at 2022-11-28 23:48:52 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container test-webserver ready: true, restart count 0 Nov 28 23:49:19.302: INFO: hostexec-ip-172-20-51-62.us-west-2.compute.internal-hmbmq started at 2022-11-28 23:49:18 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container agnhost-container ready: false, restart count 0 Nov 28 23:49:19.302: INFO: netserver-2 started at 2022-11-28 23:48:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container webserver ready: true, restart count 0 Nov 28 23:49:19.302: INFO: private started at 2022-11-28 23:48:58 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.302: INFO: Container cntr ready: true, restart count 0 Nov 28 23:49:19.616: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:49:19.616: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:49:19.682: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 44395 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6940":"ip-172-20-59-24.us-west-2.compute.internal","ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:49:19.683: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:49:19.753: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:49:19.823: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.823: INFO: Container coredns ready: true, restart count 0 Nov 28 23:49:19.823: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:49:19.823: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:49:19.823: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:49:19.823: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:19.823: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:48:46 +0000 UTC (0+7 container statuses recorded) Nov 28 23:49:19.823: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:19.823: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:19.823: INFO: netserver-3 started at 2022-11-28 23:48:55 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:19.823: INFO: Container webserver ready: true, restart count 0 Nov 28 23:49:20.174: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:49:20.174: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:49:20.240: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 42158 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:49:20.240: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:49:20.309: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:49:20.385: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:49:20.386: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:49:20.386: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:49:20.386: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:49:20.386: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:49:20.386: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:49:20.386: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:49:20.386: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:49:20.386: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:49:20.386: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:49:20.386: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:49:20.386: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:49:20.386: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:49:20.386: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:49:20.386: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:49:20.386: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:49:20.662: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:49:20.662: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-5643" for this suite.
Find pod-e4f12b0b-f4ea-4f1a-a91c-50d400605cef mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:48:38.745: Unexpected error: <*errors.errorString | 0xc003fa0510>: { s: "pod \"pod-e57f2856-c684-415a-97d0-a78afc779408\" is not Running: timed out waiting for the condition", } pod "pod-e57f2856-c684-415a-97d0-a78afc779408" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:43:37.970: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:43:38.291: INFO: Creating resource for dynamic PV Nov 28 23:43:38.291: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-8349-e2e-scjxvjg �[1mSTEP�[0m: creating a claim Nov 28 23:43:38.357: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-8349 with fsgroup 1000 Nov 28 23:48:38.745: FAIL: Unexpected error: <*errors.errorString | 0xc003fa0510>: { s: "pod \"pod-e57f2856-c684-415a-97d0-a78afc779408\" is not Running: timed out waiting for the condition", } pod "pod-e57f2856-c684-415a-97d0-a78afc779408" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc003d59a20, 0xc0005a90d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000227080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000227080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000227080, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 28 23:48:38.809: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comfqtc9" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-8349". �[1mSTEP�[0m: Found 5 events. Nov 28 23:48:39.001: INFO: At 2022-11-28 23:43:38 +0000 UTC - event for ebs.csi.aws.comfqtc9: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:48:39.001: INFO: At 2022-11-28 23:43:38 +0000 UTC - event for ebs.csi.aws.comfqtc9: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:48:39.001: INFO: At 2022-11-28 23:43:38 +0000 UTC - event for ebs.csi.aws.comfqtc9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-8349/ebs.csi.aws.comfqtc9" Nov 28 23:48:39.001: INFO: At 2022-11-28 23:43:48 +0000 UTC - event for ebs.csi.aws.comfqtc9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8349-e2e-scjxvjg": rpc error: code = Internal desc = Could not create volume "pvc-18fa4636-66cf-4ac7-9a83-7673e709ddff": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:48:39.001: INFO: At 2022-11-28 23:43:59 +0000 UTC - event for ebs.csi.aws.comfqtc9: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-8349-e2e-scjxvjg": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:48:39.065: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:48:39.065: INFO: pod-e57f2856-c684-415a-97d0-a78afc779408 Pending [] Nov 28 23:48:39.065: INFO: Nov 28 23:48:39.129: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:48:39.192: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 43718 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:48:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:48:13 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:48:39.193: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:48:39.258: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:48:39.334: INFO: test-rolling-update-with-lb-5f85c5c5f4-64kfg started at 2022-11-28 23:47:35 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.334: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:48:39.334: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-6wk67 started at 2022-11-28 23:48:30 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.334: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:48:39.334: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:48:39.334: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:48:39.334: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:48:39.334: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:48:39.334: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.334: INFO: Container coredns ready: true, restart count 0 Nov 28 23:48:39.334: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:48:39.334: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:48:39.334: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:48:39.334: INFO: netserver-0 started at 2022-11-28 23:48:19 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.334: INFO: Container webserver ready: false, restart count 0 Nov 28 23:48:39.334: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.334: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:48:39.642: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:48:39.642: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:48:39.705: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 38659 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:48:39.706: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:48:39.771: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:48:39.837: INFO: pod-ephm-test-projected-hcq2 started at 2022-11-28 23:47:33 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.837: INFO: Container test-container-subpath-projected-hcq2 ready: false, restart count 0 Nov 28 23:48:39.837: INFO: pfpod started at 2022-11-28 23:48:38 +0000 UTC (0+2 container statuses recorded) Nov 28 23:48:39.837: INFO: Container portforwardtester ready: false, restart count 0 Nov 28 23:48:39.837: INFO: Container readiness ready: false, restart count 0 Nov 28 23:48:39.837: INFO: test-rolling-update-with-lb-5f85c5c5f4-nbdrn started at 2022-11-28 23:47:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.837: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:48:39.837: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:48:39.837: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:48:39.837: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:48:39.837: INFO: netserver-1 started at 2022-11-28 23:48:19 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:39.837: INFO: Container webserver ready: false, restart count 0 Nov 28 23:48:39.837: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:48:39.837: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:48:39.837: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:48:39.837: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:48:40.088: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:48:40.089: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:48:40.154: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 41438 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:46:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:48:40.154: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:48:40.220: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:48:40.286: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:48:40.286: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:48:40.286: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:48:40.286: INFO: liveness-98271573-f7d8-4da0-a6a6-7d2ee626de6f started at 2022-11-28 23:44:43 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.286: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:48:40.286: INFO: webserver-pod started at 2022-11-28 23:46:39 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.286: INFO: Container agnhost ready: false, restart count 0 Nov 28 23:48:40.286: INFO: agnhost-primary-fdb4l started at 2022-11-28 23:48:36 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.286: INFO: Container agnhost-primary ready: true, restart count 0 Nov 28 23:48:40.286: INFO: test-ss-0 started at 2022-11-28 23:48:33 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.286: INFO: Container webserver ready: true, restart count 0 Nov 28 23:48:40.286: INFO: netserver-2 started at 2022-11-28 23:48:19 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.286: INFO: Container webserver ready: true, restart count 0 Nov 28 23:48:40.286: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:48:40.286: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:48:40.286: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:48:40.286: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:48:40.551: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:48:40.551: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:48:40.615: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 43891 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:48:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:48:40.615: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:48:40.681: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:48:40.747: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.747: INFO: Container coredns ready: true, restart count 0 Nov 28 23:48:40.747: INFO: test-rolling-update-with-lb-5f85c5c5f4-ffpvb started at 2022-11-28 23:47:33 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.747: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:48:40.747: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:48:40.747: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:48:40.747: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:48:40.747: INFO: hostexec-ip-172-20-59-24.us-west-2.compute.internal-h5tlq started at 2022-11-28 23:48:37 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.747: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:48:40.747: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:48:40.747: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:48:40.747: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:48:40.747: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:48:40.747: INFO: netserver-3 started at 2022-11-28 23:48:19 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:40.747: INFO: Container webserver ready: true, restart count 0 Nov 28 23:48:41.028: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:48:41.028: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:48:41.092: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 42158 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:47:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:48:41.092: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:48:41.158: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:48:41.226: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:48:41.226: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:48:41.226: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:48:41.226: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:48:41.226: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:48:41.226: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:48:41.226: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:48:41.226: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:48:41.226: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:48:41.226: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:48:41.226: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:48:41.226: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:48:41.226: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:48:41.226: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:48:41.226: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:48:41.226: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:48:41.535: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:48:41.535: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-8349" for this suite.
Find pod-e57f2856-c684-415a-97d0-a78afc779408 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sskips\sownership\schanges\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:53:02.772: Unexpected error: <*errors.errorString | 0xc0021dc8a0>: { s: "pod \"pod-eb91ee35-8124-4a47-a54b-932238d9b34e\" is not Running: timed out waiting for the condition", } pod "pod-eb91ee35-8124-4a47-a54b-932238d9b34e" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250from junit_18.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:48:01.976: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208 Nov 28 23:48:02.306: INFO: Creating resource for dynamic PV Nov 28 23:48:02.306: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-7146-e2e-scfrtl5 �[1mSTEP�[0m: creating a claim Nov 28 23:48:02.373: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-7146 with fsgroup 1000 Nov 28 23:53:02.771: FAIL: Unexpected error: <*errors.errorString | 0xc0021dc8a0>: { s: "pod \"pod-eb91ee35-8124-4a47-a54b-932238d9b34e\" is not Running: timed out waiting for the condition", } pod "pod-eb91ee35-8124-4a47-a54b-932238d9b34e" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc002ae5600, 0xc0050ef0d8, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:250 +0x1c5 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:221 +0x219 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000d52780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000d52780) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000d52780, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 �[1mSTEP�[0m: Deleting pvc Nov 28 23:53:02.838: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comnlw4s" �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-7146". �[1mSTEP�[0m: Found 4 events. Nov 28 23:53:03.040: INFO: At 2022-11-28 23:48:02 +0000 UTC - event for ebs.csi.aws.comnlw4s: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:53:03.040: INFO: At 2022-11-28 23:48:02 +0000 UTC - event for ebs.csi.aws.comnlw4s: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:53:03.040: INFO: At 2022-11-28 23:48:02 +0000 UTC - event for ebs.csi.aws.comnlw4s: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-7146/ebs.csi.aws.comnlw4s" Nov 28 23:53:03.040: INFO: At 2022-11-28 23:48:12 +0000 UTC - event for ebs.csi.aws.comnlw4s: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "fsgroupchangepolicy-7146-e2e-scfrtl5": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:53:03.105: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:53:03.105: INFO: pod-eb91ee35-8124-4a47-a54b-932238d9b34e Pending [] Nov 28 23:53:03.105: INFO: Nov 28 23:53:03.172: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:53:03.237: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 48325 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:51:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:51:33 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:53:03.237: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:53:03.305: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:53:03.373: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:03.373: INFO: Container coredns ready: true, restart count 0 Nov 28 23:53:03.373: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:53:03.373: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:53:03.373: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:53:03.373: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:53:03.373: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:53:03.373: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:53:03.373: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:53:03.373: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:03.373: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:53:03.680: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:53:03.680: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:53:03.745: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 47731 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:49:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:53:03.746: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:53:03.813: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:53:03.881: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:53:03.881: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:53:03.881: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:53:03.881: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:53:03.881: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:53:03.881: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:53:03.881: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:53:04.136: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:53:04.136: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:53:04.201: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 48229 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/sample-apiserver@sha256:e7fddbaac4c3451da2365ab90bad149d32f11409738034e41e0f460927f7c276 k8s.gcr.io/e2e-test-images/sample-apiserver:1.17.4],SizeBytes:24757245,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:53:04.202: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:53:04.269: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:53:04.336: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:53:04.337: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:53:04.337: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:53:04.337: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:53:04.337: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:53:04.337: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:53:04.337: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:53:04.756: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:53:04.756: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:53:04.822: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 48408 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0397ee12339c3d25d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:51:34 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.24,},NodeAddress{Type:ExternalIP,Address:54.244.71.184,},NodeAddress{Type:Hostname,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-59-24.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-244-71-184.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec209633132aaa2e2c8734b5fab5b480,SystemUUID:ec209633-132a-aa2e-2c87-34b5fab5b480,BootID:1f77a2d1-e6ae-41b0-b7d3-020128376d81,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6820^2d6a9fd0-6f76-11ed-afab-16e35050c97f],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:53:04.822: INFO: Logging kubelet events for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:53:04.890: INFO: Logging pods the kubelet thinks is on node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:53:04.959: INFO: ebs-csi-node-fgnl7 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:53:04.959: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:53:04.959: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:53:04.959: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:53:04.959: INFO: coredns-5fcc7b6498-k58g7 started at 2022-11-28 23:17:16 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:04.959: INFO: Container coredns ready: true, restart count 0 Nov 28 23:53:04.959: INFO: cilium-lv8nn started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:53:04.959: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:53:04.959: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:53:05.252: INFO: Latency metrics for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:53:05.252: INFO: Logging node info for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:53:05.318: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-60-196.us-west-2.compute.internal 465932ec-6dab-4718-b00f-11b4b1ac6741 48526 0 2022-11-28 23:14:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-60-196.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-08b36222a7b2fa9bb"} io.cilium.network.ipv4-cilium-host:172.20.34.40 io.cilium.network.ipv4-health-ip:172.20.39.176 io.cilium.network.ipv4-pod-cidr:100.96.0.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-28 23:14:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2022-11-28 23:15:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2022-11-28 23:15:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:15:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {cilium-agent Update v1 2022-11-28 23:16:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:16:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-08b36222a7b2fa9bb,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892310016 0} {<nil>} 3801084Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787452416 0} {<nil>} 3698684Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:16 +0000 UTC,LastTransitionTime:2022-11-28 23:16:16 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:52:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:52:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:52:11 +0000 UTC,LastTransitionTime:2022-11-28 23:14:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:52:11 +0000 UTC,LastTransitionTime:2022-11-28 23:15:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.60.196,},NodeAddress{Type:ExternalIP,Address:54.149.245.58,},NodeAddress{Type:Hostname,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-60-196.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-149-245-58.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fe06691704f206c05435521295741,SystemUUID:ec2fe066-9170-4f20-6c05-435521295741,BootID:9a9fa8dd-7c13-4cf7-a2a5-557158c81470,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:a91fdaf9b988943a9c73d422348c2383c08dfc2566d4124a39a1b3d785018720 registry.k8s.io/etcdadm/etcd-manager:v3.0.20220831],SizeBytes:216515768,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.22.16],SizeBytes:129618394,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.22.16],SizeBytes:123306295,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.22.16],SizeBytes:53956920,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.26.0-alpha.2],SizeBytes:42909643,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.26.0-alpha.2],SizeBytes:41441804,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[quay.io/cilium/operator@sha256:e2b26376a904190c73498e04bd5c1298a4c5239d50feab024074e4516381890f quay.io/cilium/operator:v1.11.11],SizeBytes:24003684,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.26.0-alpha.2],SizeBytes:4955300,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:53:05.318: INFO: Logging kubelet events for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:53:05.386: INFO: Logging pods the kubelet thinks is on node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:53:05.457: INFO: cilium-grpz2 started at 2022-11-28 23:15:13 +0000 UTC (1+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:53:05.457: INFO: kops-controller-qz4f4 started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container kops-controller ready: true, restart count 0 Nov 28 23:53:05.457: INFO: ebs-csi-controller-764b64d679-m7rnp started at 2022-11-28 23:15:13 +0000 UTC (0+5 container statuses recorded) Nov 28 23:53:05.457: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:53:05.457: INFO: etcd-manager-events-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:53:05.457: INFO: etcd-manager-main-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container etcd-manager ready: true, restart count 0 Nov 28 23:53:05.457: INFO: kube-controller-manager-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container kube-controller-manager ready: true, restart count 2 Nov 28 23:53:05.457: INFO: dns-controller-5fb778d9d5-fbsjl started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container dns-controller ready: true, restart count 0 Nov 28 23:53:05.457: INFO: cilium-operator-587dd66f87-4phmb started at 2022-11-28 23:15:13 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container cilium-operator ready: true, restart count 0 Nov 28 23:53:05.457: INFO: kube-apiserver-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+2 container statuses recorded) Nov 28 23:53:05.457: INFO: Container healthcheck ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container kube-apiserver ready: true, restart count 1 Nov 28 23:53:05.457: INFO: kube-scheduler-ip-172-20-60-196.us-west-2.compute.internal started at 2022-11-28 23:14:02 +0000 UTC (0+1 container statuses recorded) Nov 28 23:53:05.457: INFO: Container kube-scheduler ready: true, restart count 0 Nov 28 23:53:05.457: INFO: ebs-csi-node-7rd6q started at 2022-11-28 23:15:13 +0000 UTC (0+3 container statuses recorded) Nov 28 23:53:05.457: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:53:05.457: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:53:05.722: INFO: Latency metrics for node ip-172-20-60-196.us-west-2.compute.internal Nov 28 23:53:05.722: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-7146" for this suite.
Find pod-eb91ee35-8124-4a47-a54b-932238d9b34e mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\smount\soptions$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 28 23:44:31.519: Unexpected error: <*errors.errorString | 0xc001012bb0>: { s: "pod \"pod-224909f0-3694-48bc-9873-7a906235412d\" is not Running: timed out waiting for the condition", } pod "pod-224909f0-3694-48bc-9873-7a906235412d" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Nov 28 23:39:30.614: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace [It] should provision storage with mount options /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Nov 28 23:39:30.935: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 28 23:39:30.935: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 28 23:39:30.935: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-8265-e2e-scnjgzk 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-8265 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-8265-e2e-scnjgzk,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-8265 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-8265-e2e-scnjgzk,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-8265 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-8265-e2e-scnjgzk,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-8265-e2e-scnjgzk e6cffd4e-cf86-4109-b572-c93918c2c30e 27072 0 2022-11-28 23:39:31 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-11-28 23:39:31 +0000 UTC FieldsV1 {"f:allowVolumeExpansion":{},"f:mountOptions":{},"f:parameters":{".":{},"f:encrypted":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:*Delete,MountOptions:[dirsync],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-gqb69 pvc- provisioning-8265 c8a6cbed-6bed-423b-9c18-3020e4c5e541 27083 0 2022-11-28 23:39:31 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-11-28 23:39:31 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-8265-e2e-scnjgzk,VolumeMode:*Filesystem,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} Nov 28 23:44:31.519: FAIL: Unexpected error: <*errors.errorString | 0xc001012bb0>: { s: "pod \"pod-224909f0-3694-48bc-9873-7a906235412d\" is not Running: timed out waiting for the condition", } pod "pod-224909f0-3694-48bc-9873-7a906235412d" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning(0x7973fe8, 0xc00302dce0, 0xc002fef6c0, 0xc002f381c0, 0xc002f38380, 0xc003d67500, 0x0, 0x0, 0x0, 0x0, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418 +0xe7d k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:198 +0x245 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000a37c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x36c k8s.io/kubernetes/test/e2e.TestE2E(0xc000a37c80) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b testing.tRunner(0xc000a37c80, 0x729c8b8) /usr/local/go/src/testing/testing.go:1203 +0xe5 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1248 +0x2b3 Nov 28 23:44:31.519: INFO: deleting claim "provisioning-8265"/"pvc-gqb69" Nov 28 23:44:31.584: INFO: deleting storage class provisioning-8265-e2e-scnjgzk [AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "provisioning-8265". �[1mSTEP�[0m: Found 5 events. Nov 28 23:44:31.713: INFO: At 2022-11-28 23:39:31 +0000 UTC - event for pvc-gqb69: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Nov 28 23:44:31.713: INFO: At 2022-11-28 23:39:31 +0000 UTC - event for pvc-gqb69: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Nov 28 23:44:31.713: INFO: At 2022-11-28 23:39:31 +0000 UTC - event for pvc-gqb69: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } Provisioning: External provisioner is provisioning volume for claim "provisioning-8265/pvc-gqb69" Nov 28 23:44:31.713: INFO: At 2022-11-28 23:39:41 +0000 UTC - event for pvc-gqb69: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "provisioning-8265-e2e-scnjgzk": rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 28 23:44:31.713: INFO: At 2022-11-28 23:40:04 +0000 UTC - event for pvc-gqb69: {ebs.csi.aws.com_ebs-csi-controller-764b64d679-m7rnp_571dba60-e642-4557-920e-a7be04aaa84d } ProvisioningFailed: failed to provision volume with StorageClass "provisioning-8265-e2e-scnjgzk": rpc error: code = Internal desc = Could not create volume "pvc-c8a6cbed-6bed-423b-9c18-3020e4c5e541": could not create volume in EC2: RequestCanceled: request context canceled caused by: context deadline exceeded Nov 28 23:44:31.776: INFO: POD NODE PHASE GRACE CONDITIONS Nov 28 23:44:31.776: INFO: pod-224909f0-3694-48bc-9873-7a906235412d Pending [] Nov 28 23:44:31.776: INFO: Nov 28 23:44:31.841: INFO: Logging node info for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:31.904: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-9.us-west-2.compute.internal b164bdf4-747d-40a4-a0e2-c107faba792c 34218 0 2022-11-28 23:16:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-9.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node mounted_volume_expand:mounted-volume-expand-2213 node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-33-9.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-079ec6164b883508f"} io.cilium.network.ipv4-cilium-host:172.20.62.7 io.cilium.network.ipv4-health-ip:172.20.33.101 io.cilium.network.ipv4-pod-cidr:100.96.1.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2022-11-28 23:16:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {e2e.test Update v1 2022-11-28 23:35:50 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:mounted_volume_expand":{}}}} } {kubelet Update v1 2022-11-28 23:41:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-079ec6164b883508f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:59 +0000 UTC,LastTransitionTime:2022-11-28 23:16:59 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:41:42 +0000 UTC,LastTransitionTime:2022-11-28 23:16:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.9,},NodeAddress{Type:ExternalIP,Address:35.87.131.193,},NodeAddress{Type:Hostname,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-87-131-193.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c2c6bcc0076f6d707129f45de46db,SystemUUID:ec2c2c6b-cc00-76f6-d707-129f45de46db,BootID:cd503269-ab9d-47d3-b010-5ad05f73aef2,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:aa60f453d64dfb3c3fd9e7306f988c36c6352c4f2d956aa90467f2808091effa registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.5],SizeBytes:15209420,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:31.905: INFO: Logging kubelet events for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:31.971: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:32.038: INFO: netserver-0 started at 2022-11-28 23:44:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:32.038: INFO: test-rolling-update-with-lb-7db9cf5ff8-bjp4c started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:32.038: INFO: startup-adb08a1a-8594-4c57-8938-88cc08e1a18f started at 2022-11-28 23:41:11 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container busybox ready: false, restart count 0 Nov 28 23:44:32.038: INFO: cilium-q4f96 started at 2022-11-28 23:16:32 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:32.038: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:32.038: INFO: coredns-autoscaler-6658b4bf85-sstkf started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container autoscaler ready: true, restart count 0 Nov 28 23:44:32.038: INFO: netserver-0 started at 2022-11-28 23:44:04 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:32.038: INFO: hostexec-ip-172-20-33-9.us-west-2.compute.internal-z4hp8 started at 2022-11-28 23:44:29 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:44:32.038: INFO: ebs-csi-node-ztm7q started at 2022-11-28 23:16:32 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:32.038: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:32.038: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:32.038: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:32.038: INFO: coredns-5fcc7b6498-422r7 started at 2022-11-28 23:16:51 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.038: INFO: Container coredns ready: true, restart count 0 Nov 28 23:44:32.327: INFO: Latency metrics for node ip-172-20-33-9.us-west-2.compute.internal Nov 28 23:44:32.327: INFO: Logging node info for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:32.390: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-43-89.us-west-2.compute.internal 3c1a0bd0-7a39-456f-b30e-f2c3b500d78e 38659 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-43-89.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-43-89.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05e37988a8cbf1294"} io.cilium.network.ipv4-cilium-host:172.20.35.202 io.cilium.network.ipv4-health-ip:172.20.46.111 io.cilium.network.ipv4-pod-cidr:100.96.2.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-28 23:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-05e37988a8cbf1294,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:16:58 +0000 UTC,LastTransitionTime:2022-11-28 23:16:58 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:04 +0000 UTC,LastTransitionTime:2022-11-28 23:16:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.89,},NodeAddress{Type:ExternalIP,Address:54.245.165.148,},NodeAddress{Type:Hostname,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-43-89.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-245-165-148.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24fb960e77dda4c1984f64b4ac46aa,SystemUUID:ec24fb96-0e77-dda4-c198-4f64b4ac46aa,BootID:fb3609df-79b8-4850-bb18-8ff0184f0e67,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9892^5f85df7d-6f75-11ed-90b2-e2f74bcf6311],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 28 23:44:32.391: INFO: Logging kubelet events for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:32.456: INFO: Logging pods the kubelet thinks is on node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:32.524: INFO: netserver-1 started at 2022-11-28 23:44:08 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:32.524: INFO: condition-test-b68v6 started at 2022-11-28 23:44:28 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Container httpd ready: true, restart count 0 Nov 28 23:44:32.524: INFO: test-container-pod started at 2022-11-28 23:44:31 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Container webserver ready: false, restart count 0 Nov 28 23:44:32.524: INFO: cilium-swmmt started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:32.524: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:32.524: INFO: ebs-csi-node-sdlj6 started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:32.524: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:32.524: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:32.524: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:32.524: INFO: test-rolling-update-with-lb-7db9cf5ff8-fd9gb started at 2022-11-28 23:44:20 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Container agnhost ready: true, restart count 0 Nov 28 23:44:32.524: INFO: netserver-1 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:32.524: INFO: pod-projected-configmaps-38f797bb-63d1-4124-a565-c4ab4299c7a1 started at 2022-11-28 23:44:22 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:32.524: INFO: Container agnhost-container ready: true, restart count 0 Nov 28 23:44:32.850: INFO: Latency metrics for node ip-172-20-43-89.us-west-2.compute.internal Nov 28 23:44:32.850: INFO: Logging node info for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:32.913: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-51-62.us-west-2.compute.internal 7c4b2ae2-5298-46e1-8ece-9644d06d3f0c 38663 0 2022-11-28 23:16:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-51-62.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-51-62.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-217":"ip-172-20-51-62.us-west-2.compute.internal","ebs.csi.aws.com":"i-043b6631a25f30b05"} io.cilium.network.ipv4-cilium-host:172.20.44.93 io.cilium.network.ipv4-health-ip:172.20.37.18 io.cilium.network.ipv4-pod-cidr:100.96.4.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:17:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:44:14 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:44:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-043b6631a25f30b05,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-28 23:17:00 +0000 UTC,LastTransitionTime:2022-11-28 23:17:00 +0000 UTC,Reason:CiliumIsUp,Message:Cilium is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:32 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-28 23:44:14 +0000 UTC,LastTransitionTime:2022-11-28 23:16:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.62,},NodeAddress{Type:ExternalIP,Address:54.186.219.113,},NodeAddress{Type:Hostname,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-51-62.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-186-219-113.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2bccc9d2e6b93f9422fbbc73278b46,SystemUUID:ec2bccc9-d2e6-b93f-9422-fbbc73278b46,BootID:cc1f95d2-6e86-402f-b028-49d2003097ab,KernelVersion:5.15.0-1023-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.13,KubeletVersion:v1.22.16,KubeProxyVersion:v1.22.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:fad1627d1c52ff084c3147bbeba7a7b64c60562db38c5e7a9b37c710a8f1d3c1 quay.io/cilium/cilium:v1.11.11],SizeBytes:160382894,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:702a992280fb7c3303e84a5801acbb4c9c7fcf48cffe0e9c8be3f0c60f74cf89 k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.4],SizeBytes:112029652,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.22.16],SizeBytes:105446599,},ContainerImage{Names:[docker.io/library/nginx@sha256:e209ac2f37c70c1e0e9873a5f7231e91dcd83fdf1178d8ed36c2ec09974210ba docker.io/library/nginx:latest],SizeBytes:56833911,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:51105200,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:1f36a24cfb5e0c3f725d7565a867c2384282fcbeccc77b07b423c9da95763a9a k8s.gcr.io/e2e-test-images/nautilus:1.4],SizeBytes:49230179,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:716d2f68314c5c4ddd5ecdb45183fcb4ed8019015982c1321571f863989b70b0 k8s.gcr.io/e2e-test-images/httpd:2.4.39-1],SizeBytes:41902332,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:b913fa234cc3473cfe16e937d106b455a7609f927f59031c81aca791e2689b50 k8s.gcr.io/e2e-test-images/httpd:2.4.38-1],SizeBytes:40765006,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:21c6eedc2a80bfec5dee669d30d9b5ca83e68cd097f0e210917ae91312e590ff registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.12.0],SizeBytes:28956694,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:22631062,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:22629806,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:21584611,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:21367429,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:21366448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:21331336,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonroot@sha256:4051e85640c22f8e00c34dbd273576fc9e1e2829992656588062be9c0f69b04b k8s.gcr.io/e2e-test-images/nonroot:1.1],SizeBytes:17748448,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:14930811,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:8561694,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:8223849,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:1b7c978a792a8fa4e96244e8059bd71bb49b07e2e5a897fb0c867bdc6db20d5d k8s.gcr.io/sig-storage/livenessprobe:v2.3.0],SizeBytes:7933739,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:6979365,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:1ff6c18fbef2045af6b9c16bf034cc421a29027b800e4f9b68ae9b1cb3e9ae07 k8s.gcr.io/pause:3.5],SizeBytes:301416,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-217^8ff49ff7-6f76-11ed-af79-e248a53deb4d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-217^8ff49ff7-6f76-11ed-af79-e248a53deb4d,DevicePath:,},},Config:nil,},} Nov 28 23:44:32.914: INFO: Logging kubelet events for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:32.979: INFO: Logging pods the kubelet thinks is on node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:33.045: INFO: cilium-j5vcb started at 2022-11-28 23:16:33 +0000 UTC (1+1 container statuses recorded) Nov 28 23:44:33.045: INFO: Init container clean-cilium-state ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container cilium-agent ready: true, restart count 0 Nov 28 23:44:33.045: INFO: csi-hostpathplugin-0 started at 2022-11-28 23:44:08 +0000 UTC (0+7 container statuses recorded) Nov 28 23:44:33.045: INFO: Container csi-attacher ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container csi-provisioner ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container csi-resizer ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container hostpath ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:33.045: INFO: ebs-csi-node-btmzq started at 2022-11-28 23:16:33 +0000 UTC (0+3 container statuses recorded) Nov 28 23:44:33.045: INFO: Container ebs-plugin ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container liveness-probe ready: true, restart count 0 Nov 28 23:44:33.045: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 28 23:44:33.045: INFO: netserver-2 started at 2022-11-28 23:44:05 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.045: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:33.045: INFO: netserver-2 started at 2022-11-28 23:44:09 +0000 UTC (0+1 container statuses recorded) Nov 28 23:44:33.045: INFO: Container webserver ready: true, restart count 0 Nov 28 23:44:33.411: INFO: Latency metrics for node ip-172-20-51-62.us-west-2.compute.internal Nov 28 23:44:33.411: INFO: Logging node info for node ip-172-20-59-24.us-west-2.compute.internal Nov 28 23:44:33.474: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-59-24.us-west-2.compute.internal 7c344817-f6e4-40fe-898e-7f1473ba3530 37644 0 2022-11-28 23:16:32 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-59-24.us-west-2.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:ip-172-20-59-24.us-west-2.compute.internal topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0397ee12339c3d25d"} io.cilium.network.ipv4-cilium-host:172.20.58.211 io.cilium.network.ipv4-health-ip:172.20.36.181 io.cilium.network.ipv4-pod-cidr:100.96.3.0/24 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-28 23:16:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cilium-agent Update v1 2022-11-28 23:16:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:io.cilium.network.ipv4-cilium-host":{},"f:io.cilium.network.ipv4-health-ip":{},"f:io.cilium.network.ipv4-pod-cidr":{}}}} status} {cilium-operator Update v1 2022-11-28 23:16:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-28 23:41:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-28 23:41:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone"