e2e-kops-aws-upgrade-kstable-kolatest-to-klatest-kolatest #1639401656711909376
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 269 failed / 647 succeeded |
| Started | |
| Elapsed | 59m42s |
| Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sexisting\sdirectories\swhen\sreadOnly\sspecified\sin\sthe\svolumeSource$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-fvdk for test-container-subpath-dynamicpv-fvdk: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-318/pods/pod-subpath-test-dynamicpv-fvdk/log?container=test-container-subpath-dynamicpv-fvdk&previous=false": stream error: stream ID 1641; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:46:21.212from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:46:01.457 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:46:01.457 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:46:01.457 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:46:01.457 Mar 24 23:46:01.457: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:46:01.458 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:46:01.656 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:46:01.784 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:46:01.913 (456ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:46:01.913 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:46:01.913 (0s) > Enter [It] should support existing directories when readOnly specified in the volumeSource - test/e2e/storage/testsuites/subpath.go:397 @ 03/24/23 23:46:01.913 Mar 24 23:46:01.913: INFO: Creating resource for dynamic PV Mar 24 23:46:01.913: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-318-e2e-scwdjsm - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:46:01.914 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:46:01.985 Mar 24 23:46:01.985: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-fvdk - test/e2e/storage/testsuites/subpath.go:691 @ 03/24/23 23:46:02.121 STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:46:02.121 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:46:20.939 Mar 24 23:46:21.008: INFO: Trying to get logs from node i-0cdea7500a633a81f pod pod-subpath-test-dynamicpv-fvdk container test-container-subpath-dynamicpv-fvdk: <nil> Mar 24 23:46:21.072: INFO: Failed to get logs from node "i-0cdea7500a633a81f" pod "pod-subpath-test-dynamicpv-fvdk" container "test-container-subpath-dynamicpv-fvdk". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-318/pods/pod-subpath-test-dynamicpv-fvdk/log?container=test-container-subpath-dynamicpv-fvdk&previous=false": stream error: stream ID 1641; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:46:21.072 Mar 24 23:46:21.211: INFO: Unexpected error: <*fmt.wrapError | 0xc004bd8d40>: failed to get logs from pod-subpath-test-dynamicpv-fvdk for test-container-subpath-dynamicpv-fvdk: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-318/pods/pod-subpath-test-dynamicpv-fvdk/log?container=test-container-subpath-dynamicpv-fvdk&previous=false": stream error: stream ID 1641; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-fvdk for test-container-subpath-dynamicpv-fvdk: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-318/pods/pod-subpath-test-dynamicpv-fvdk/log?container=test-container-subpath-dynamicpv-fvdk&previous=false\": stream error: stream ID 1641; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc004733ce0>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-318/pods/pod-subpath-test-dynamicpv-fvdk/log?container=test-container-subpath-dynamicpv-fvdk&previous=false", Err: <http2.StreamError>{ StreamID: 1641, Code: 2, Cause: <*errors.errorString | 0xc0001676a0>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-fvdk for test-container-subpath-dynamicpv-fvdk: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-318/pods/pod-subpath-test-dynamicpv-fvdk/log?container=test-container-subpath-dynamicpv-fvdk&previous=false": stream error: stream ID 1641; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:46:21.212 < Exit [It] should support existing directories when readOnly specified in the volumeSource - test/e2e/storage/testsuites/subpath.go:397 @ 03/24/23 23:46:21.212 (19.299s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:46:21.212 Mar 24 23:46:21.212: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:46:21.278 (66ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:399 @ 03/24/23 23:46:21.278 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:46:21.278 Mar 24 23:46:21.278: INFO: Deleting pod "pod-subpath-test-dynamicpv-fvdk" in namespace "provisioning-318" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:46:21.343 Mar 24 23:46:21.472: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comp6msz" Mar 24 23:46:21.540: INFO: Waiting up to 5m0s for PersistentVolume pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826 to get deleted Mar 24 23:46:21.605: INFO: PersistentVolume pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826 found and phase=Released (64.882576ms) Mar 24 23:46:26.673: INFO: PersistentVolume pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826 found and phase=Released (5.132821048s) Mar 24 23:46:31.739: INFO: PersistentVolume pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826 found and phase=Released (10.199245282s) Mar 24 23:46:36.809: INFO: PersistentVolume pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826 was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:46:36.809 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:399 @ 03/24/23 23:46:36.883 (15.605s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:46:36.883 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:46:36.884 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:46:36.884 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:46:36.884 STEP: Collecting events from namespace "provisioning-318". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:46:36.884 STEP: Found 15 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:46:36.949 Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:02 +0000 UTC - event for ebs.csi.aws.comp6msz: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:02 +0000 UTC - event for ebs.csi.aws.comp6msz: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-318/ebs.csi.aws.comp6msz" Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:02 +0000 UTC - event for ebs.csi.aws.comp6msz: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:05 +0000 UTC - event for ebs.csi.aws.comp6msz: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826 Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:06 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {default-scheduler } Scheduled: Successfully assigned provisioning-318/pod-subpath-test-dynamicpv-fvdk to i-0cdea7500a633a81f Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:08 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-9f0e1cda-f914-4e75-b0cf-d9e48279d826" Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:14 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:14 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Created: Created container init-volume-dynamicpv-fvdk Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:15 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Started: Started container init-volume-dynamicpv-fvdk Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:15 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:15 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Created: Created container test-init-subpath-dynamicpv-fvdk Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:15 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Started: Started container test-init-subpath-dynamicpv-fvdk Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:16 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:16 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Created: Created container test-container-subpath-dynamicpv-fvdk Mar 24 23:46:36.949: INFO: At 2023-03-24 23:46:16 +0000 UTC - event for pod-subpath-test-dynamicpv-fvdk: {kubelet i-0cdea7500a633a81f} Started: Started container test-container-subpath-dynamicpv-fvdk Mar 24 23:46:37.014: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:46:37.014: INFO: Mar 24 23:46:37.080: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:46:37.145: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 39454 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-capacity-6575":"i-005794773d640afd9","csi-mock-csi-mock-volumes-expansion-8579":"csi-mock-csi-mock-volumes-expansion-8579","csi-mock-csi-mock-volumes-workload-8221":"i-005794773d640afd9","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:46:26 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:46:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:30 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:30 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:30 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:46:30 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-8221^1591fc16-ca9e-11ed-8a10-ae8815acf7bc kubernetes.io/csi/ebs.csi.aws.com^vol-04ef755f98b75a651 kubernetes.io/csi/ebs.csi.aws.com^vol-0f3bc6aca398a0679],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-04ef755f98b75a651,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0f3bc6aca398a0679,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-8221^1591fc16-ca9e-11ed-8a10-ae8815acf7bc,DevicePath:,},},Config:nil,},} Mar 24 23:46:37.145: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:46:37.225: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:46:37.301: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:46:11 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:46:37.301: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:46:37.301: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:46:17 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:46:37.301: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:46:37.301: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:46:37.301: INFO: csi-mockplugin-0 started at 2023-03-24 23:46:11 +0000 UTC (0+4 container statuses recorded) Mar 24 23:46:37.301: INFO: Container busybox ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container mock ready: true, restart count 0 Mar 24 23:46:37.301: INFO: csi-mockplugin-0 started at 2023-03-24 23:44:46 +0000 UTC (0+4 container statuses recorded) Mar 24 23:46:37.301: INFO: Container busybox ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container mock ready: true, restart count 0 Mar 24 23:46:37.301: INFO: pvc-volume-tester-5gwhg started at 2023-03-24 23:46:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:46:37.301: INFO: inline-volume-wkknp started at 2023-03-24 23:46:09 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:46:37.301: INFO: csi-mockplugin-0 started at 2023-03-24 23:42:03 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:37.301: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container mock ready: true, restart count 0 Mar 24 23:46:37.301: INFO: csi-mockplugin-0 started at 2023-03-24 23:46:17 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:37.301: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container mock ready: true, restart count 0 Mar 24 23:46:37.301: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:37.301: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:46:37.301: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:46:37.301: INFO: netserver-0 started at 2023-03-24 23:46:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container webserver ready: true, restart count 0 Mar 24 23:46:37.301: INFO: test-grpc-635ed0ad-e3e7-4127-8d23-2923a1542889 started at 2023-03-24 23:46:08 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:46:37.301: INFO: netserver-0 started at 2023-03-24 23:46:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.301: INFO: Container webserver ready: false, restart count 0 Mar 24 23:46:37.632: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:46:37.632: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:46:37.697: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 39502 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-6997":"i-06956c12f825d08d9","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:46:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:21 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:21 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:21 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:46:21 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:46:37.697: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:46:37.783: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:46:37.859: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:46:37.860: INFO: busybox-a56e2e62-c057-434d-8ad2-927dd3ef3ea6 started at 2023-03-24 23:45:50 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container busybox ready: false, restart count 0 Mar 24 23:46:37.860: INFO: netserver-1 started at 2023-03-24 23:46:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container webserver ready: true, restart count 0 Mar 24 23:46:37.860: INFO: netserver-1 started at 2023-03-24 23:46:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container webserver ready: false, restart count 0 Mar 24 23:46:37.860: INFO: pod-secrets-89f39212-2f92-452d-a143-8ea5b1f722b0 started at 2023-03-24 23:43:46 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:37.860: INFO: Container creates-volume-test ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container dels-volume-test ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container upds-volume-test ready: true, restart count 0 Mar 24 23:46:37.860: INFO: csi-mockplugin-0 started at 2023-03-24 23:46:01 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:37.860: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container mock ready: true, restart count 0 Mar 24 23:46:37.860: INFO: pvc-volume-tester-hfx4d started at 2023-03-24 23:46:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:46:37.860: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:46:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:46:37.860: INFO: pfpod started at 2023-03-24 23:44:20 +0000 UTC (0+2 container statuses recorded) Mar 24 23:46:37.860: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:46:37.860: INFO: Container readiness ready: false, restart count 0 Mar 24 23:46:37.860: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:37.860: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:46:37.860: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:46:37.860: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:46:37.860: INFO: hostexec-i-06956c12f825d08d9-z2h5j started at 2023-03-24 23:46:33 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:46:37.860: INFO: pod-af4c9aa8-4e14-4c2d-a930-3f6a8254e260 started at 2023-03-24 23:46:37 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:37.860: INFO: Container write-pod ready: false, restart count 0 Mar 24 23:46:38.163: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:46:38.163: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:46:38.228: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 39538 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:46:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:46:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:31 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:31 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:31 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:46:31 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-088904af6e435f33b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-088904af6e435f33b,DevicePath:,},},Config:nil,},} Mar 24 23:46:38.228: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:46:38.302: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:46:38.376: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:38.376: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:46:38.376: INFO: netserver-2 started at 2023-03-24 23:46:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container webserver ready: true, restart count 0 Mar 24 23:46:38.376: INFO: test-webserver-1b6c3aa2-9178-410f-afd7-ae5b2aba4b48 started at 2023-03-24 23:45:41 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container test-webserver ready: false, restart count 0 Mar 24 23:46:38.376: INFO: hostexec-i-0713c0b421302f422-d4mdz started at 2023-03-24 23:46:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:46:38.376: INFO: sample-apiserver-deployment-6dfd6dfd5b-c2wm2 started at 2023-03-24 23:46:25 +0000 UTC (0+2 container statuses recorded) Mar 24 23:46:38.376: INFO: Container etcd ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Container sample-apiserver ready: false, restart count 0 Mar 24 23:46:38.376: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container coredns ready: true, restart count 0 Mar 24 23:46:38.376: INFO: test-webserver-beecf91c-d4bc-4b66-9648-32e1c7cd7922 started at 2023-03-24 23:43:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container test-webserver ready: true, restart count 0 Mar 24 23:46:38.376: INFO: netserver-2 started at 2023-03-24 23:46:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container webserver ready: false, restart count 0 Mar 24 23:46:38.376: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:46:38.376: INFO: pod-subpath-test-inlinevolume-9bcm started at 2023-03-24 23:46:34 +0000 UTC (1+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Init container init-volume-inlinevolume-9bcm ready: true, restart count 0 Mar 24 23:46:38.376: INFO: Container test-container-subpath-inlinevolume-9bcm ready: false, restart count 0 Mar 24 23:46:38.376: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.376: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:46:38.376: INFO: pfpod started at 2023-03-24 23:44:53 +0000 UTC (0+2 container statuses recorded) Mar 24 23:46:38.376: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:46:38.376: INFO: Container readiness ready: false, restart count 0 Mar 24 23:46:38.671: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:46:38.671: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:46:38.737: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 36523 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:45:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:46:38.737: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:46:38.810: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:46:38.883: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:46:38.883: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:46:38.883: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:46:38.883: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:46:38.883: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:46:38.883: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:46:38.883: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:46:38.883: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:38.883: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:46:38.883: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:46:38.883: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:46:38.883: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:46:38.883: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:46:38.883: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:46:38.883: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:46:38.883: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:46:38.883: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:46:38.883: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:38.883: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:46:39.202: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:46:39.202: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:46:39.270: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 39291 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-fsgroup-mount-5924":"csi-mock-csi-mock-volumes-fsgroup-mount-5924","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:46:24 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:46:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:24 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:24 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:46:24 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:46:24 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0cd8d340169ee4b11],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0cd8d340169ee4b11,DevicePath:,},},Config:nil,},} Mar 24 23:46:39.271: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:46:39.345: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:46:39.424: INFO: test-container-pod started at 2023-03-24 23:46:31 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Container webserver ready: true, restart count 0 Mar 24 23:46:39.424: INFO: netserver-3 started at 2023-03-24 23:46:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Container webserver ready: false, restart count 0 Mar 24 23:46:39.424: INFO: netserver-3 started at 2023-03-24 23:46:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Container webserver ready: true, restart count 0 Mar 24 23:46:39.424: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Container coredns ready: true, restart count 0 Mar 24 23:46:39.424: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:46:39.424: INFO: csi-mockplugin-0 started at 2023-03-24 23:45:47 +0000 UTC (0+4 container statuses recorded) Mar 24 23:46:39.424: INFO: Container busybox ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container mock ready: true, restart count 0 Mar 24 23:46:39.424: INFO: pod-configmaps-4faa40ec-e6e6-4f18-a7f9-fbad3cc24f7e started at 2023-03-24 23:46:35 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:39.424: INFO: Container createcm-volume-test ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container delcm-volume-test ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container updcm-volume-test ready: true, restart count 0 Mar 24 23:46:39.424: INFO: host-test-container-pod started at 2023-03-24 23:46:31 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:46:39.424: INFO: pfpod started at 2023-03-24 23:45:37 +0000 UTC (0+2 container statuses recorded) Mar 24 23:46:39.424: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:46:39.424: INFO: Container readiness ready: false, restart count 0 Mar 24 23:46:39.424: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:46:39.424: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:46:39.424: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:46:39.424: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:46:39.424: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:46:39.805: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:46:39.805 (2.922s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:46:39.805 (2.922s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:46:39.805 STEP: Destroying namespace "provisioning-318" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:46:39.805 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:46:39.971 (166ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:46:39.971 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:46:39.971 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sexisting\sdirectory$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-mcbg for test-container-volume-dynamicpv-mcbg: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2859/pods/pod-subpath-test-dynamicpv-mcbg/log?container=test-container-volume-dynamicpv-mcbg&previous=false": stream error: stream ID 1721; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:43:55.804from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:43:38.203 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:43:38.203 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:43:38.203 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:43:38.203 Mar 24 23:43:38.203: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:43:38.204 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:43:38.406 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:43:38.531 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:43:38.676 (473ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:43:38.676 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:43:38.676 (0s) > Enter [It] should support existing directory - test/e2e/storage/testsuites/subpath.go:205 @ 03/24/23 23:43:38.676 Mar 24 23:43:38.676: INFO: Creating resource for dynamic PV Mar 24 23:43:38.676: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-2859-e2e-sc7dnvm - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:43:38.676 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:43:38.739 Mar 24 23:43:38.739: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-mcbg - test/e2e/storage/testsuites/subpath.go:691 @ 03/24/23 23:43:38.868 STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:43:38.868 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:43:55.539 Mar 24 23:43:55.602: INFO: Trying to get logs from node i-06956c12f825d08d9 pod pod-subpath-test-dynamicpv-mcbg container test-container-volume-dynamicpv-mcbg: <nil> Mar 24 23:43:55.664: INFO: Failed to get logs from node "i-06956c12f825d08d9" pod "pod-subpath-test-dynamicpv-mcbg" container "test-container-volume-dynamicpv-mcbg". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2859/pods/pod-subpath-test-dynamicpv-mcbg/log?container=test-container-volume-dynamicpv-mcbg&previous=false": stream error: stream ID 1721; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:43:55.665 Mar 24 23:43:55.804: INFO: Unexpected error: <*fmt.wrapError | 0xc00094dec0>: failed to get logs from pod-subpath-test-dynamicpv-mcbg for test-container-volume-dynamicpv-mcbg: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2859/pods/pod-subpath-test-dynamicpv-mcbg/log?container=test-container-volume-dynamicpv-mcbg&previous=false": stream error: stream ID 1721; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-mcbg for test-container-volume-dynamicpv-mcbg: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2859/pods/pod-subpath-test-dynamicpv-mcbg/log?container=test-container-volume-dynamicpv-mcbg&previous=false\": stream error: stream ID 1721; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc0027453e0>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2859/pods/pod-subpath-test-dynamicpv-mcbg/log?container=test-container-volume-dynamicpv-mcbg&previous=false", Err: <http2.StreamError>{ StreamID: 1721, Code: 2, Cause: <*errors.errorString | 0xc0001676a0>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-mcbg for test-container-volume-dynamicpv-mcbg: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2859/pods/pod-subpath-test-dynamicpv-mcbg/log?container=test-container-volume-dynamicpv-mcbg&previous=false": stream error: stream ID 1721; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:43:55.804 < Exit [It] should support existing directory - test/e2e/storage/testsuites/subpath.go:205 @ 03/24/23 23:43:55.804 (17.128s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:43:55.804 Mar 24 23:43:55.804: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:43:55.877 (73ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:207 @ 03/24/23 23:43:55.877 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:43:55.877 Mar 24 23:43:55.877: INFO: Deleting pod "pod-subpath-test-dynamicpv-mcbg" in namespace "provisioning-2859" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:43:55.944 Mar 24 23:43:56.073: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.combdzxj" Mar 24 23:43:56.136: INFO: Waiting up to 5m0s for PersistentVolume pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7 to get deleted Mar 24 23:43:56.199: INFO: PersistentVolume pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7 found and phase=Released (62.575087ms) Mar 24 23:44:01.266: INFO: PersistentVolume pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7 found and phase=Released (5.13013704s) Mar 24 23:44:06.329: INFO: PersistentVolume pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7 found and phase=Released (10.193065206s) Mar 24 23:44:11.393: INFO: PersistentVolume pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7 was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:44:11.393 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:207 @ 03/24/23 23:44:11.459 (15.582s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:44:11.459 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:44:11.46 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:44:11.46 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:44:11.46 STEP: Collecting events from namespace "provisioning-2859". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:44:11.46 STEP: Found 18 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:44:11.522 Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:38 +0000 UTC - event for ebs.csi.aws.combdzxj: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:38 +0000 UTC - event for ebs.csi.aws.combdzxj: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:38 +0000 UTC - event for ebs.csi.aws.combdzxj: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-2859/ebs.csi.aws.combdzxj" Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:42 +0000 UTC - event for ebs.csi.aws.combdzxj: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7 Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:42 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {default-scheduler } Scheduled: Successfully assigned provisioning-2859/pod-subpath-test-dynamicpv-mcbg to i-06956c12f825d08d9 Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:45 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-b36f7748-7c2d-474f-b66a-9e37468c87e7" Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:50 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:50 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Created: Created container init-volume-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:50 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Started: Started container init-volume-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:51 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:51 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Created: Created container test-init-subpath-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:51 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Started: Started container test-init-subpath-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:52 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:52 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Created: Created container test-container-subpath-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:52 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Started: Started container test-container-subpath-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:52 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:52 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Created: Created container test-container-volume-dynamicpv-mcbg Mar 24 23:44:11.522: INFO: At 2023-03-24 23:43:53 +0000 UTC - event for pod-subpath-test-dynamicpv-mcbg: {kubelet i-06956c12f825d08d9} Started: Started container test-container-volume-dynamicpv-mcbg Mar 24 23:44:11.584: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:44:11.584: INFO: Mar 24 23:44:11.648: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:44:11.710: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 33537 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-capacity-6575":"i-005794773d640afd9","csi-mock-csi-mock-volumes-expansion-8518":"csi-mock-csi-mock-volumes-expansion-8518","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:44:02 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:44:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:07 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:07 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:07 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:44:07 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-8518^4 kubernetes.io/csi/ebs.csi.aws.com^vol-0e7f065eea8fab6f1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-8518^4,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0e7f065eea8fab6f1,DevicePath:,},},Config:nil,},} Mar 24 23:44:11.711: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:44:11.782: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:44:11.850: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:44:11.850: INFO: pod-5c865f62-5553-4ef2-a970-df83ebe0a627 started at 2023-03-24 23:43:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container write-pod ready: true, restart count 0 Mar 24 23:44:11.850: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:44:11.850: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:44:11.850: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:43:49 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:44:11.850: INFO: csi-mockplugin-0 started at 2023-03-24 23:42:03 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:11.850: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container mock ready: true, restart count 0 Mar 24 23:44:11.850: INFO: boom-server started at 2023-03-24 23:44:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container boom-server ready: true, restart count 0 Mar 24 23:44:11.850: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:11.850: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:44:11.850: INFO: liveness-92f7bad4-2180-4189-9432-5763c759fd4f started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:44:11.850: INFO: pvc-volume-tester-wk5rw started at 2023-03-24 23:44:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:44:11.850: INFO: csi-mockplugin-0 started at 2023-03-24 23:43:49 +0000 UTC (0+4 container statuses recorded) Mar 24 23:44:11.850: INFO: Container busybox ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:44:11.850: INFO: Container mock ready: true, restart count 0 Mar 24 23:44:11.850: INFO: test-rolling-update-with-lb-69fcddc54f-6n69z started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:44:11.850: INFO: netserver-0 started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container webserver ready: false, restart count 0 Mar 24 23:44:11.850: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:43:49 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:11.850: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:44:12.136: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:44:12.136: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:44:12.199: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 33692 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5411":"i-06956c12f825d08d9","csi-mock-csi-mock-volumes-fsgroup-policy-3108":"i-06956c12f825d08d9","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:44:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:44:11 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:43:59 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:43:59 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:43:59 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:43:59 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5411^c55158d5-ca9d-11ed-9bf3-4e47e14b5910,DevicePath:,},},Config:nil,},} Mar 24 23:44:12.199: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:44:12.274: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:44:12.382: INFO: fail-once-non-local-ng7z9 started at 2023-03-24 23:44:09 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: pod-ephm-test-configmap-gscq started at 2023-03-24 23:43:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container test-container-subpath-configmap-gscq ready: false, restart count 0 Mar 24 23:44:12.382: INFO: liveness-a407d9b9-63aa-4982-be0e-8eb1139ed722 started at 2023-03-24 23:40:58 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:44:12.382: INFO: pod-disruption-failure-ignore-0-gdl8v started at 2023-03-24 23:44:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: true, restart count 0 Mar 24 23:44:12.382: INFO: pfpod started at 2023-03-24 23:43:25 +0000 UTC (0+2 container statuses recorded) Mar 24 23:44:12.382: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:44:12.382: INFO: Container readiness ready: false, restart count 0 Mar 24 23:44:12.382: INFO: pod-secrets-89f39212-2f92-452d-a143-8ea5b1f722b0 started at 2023-03-24 23:43:46 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:12.382: INFO: Container creates-volume-test ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container dels-volume-test ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container upds-volume-test ready: true, restart count 0 Mar 24 23:44:12.382: INFO: startup-script started at 2023-03-24 23:44:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container startup-script ready: true, restart count 0 Mar 24 23:44:12.382: INFO: fail-once-non-local-9kp9s started at 2023-03-24 23:44:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: csi-mockplugin-0 started at 2023-03-24 23:43:46 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:12.382: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container mock ready: true, restart count 0 Mar 24 23:44:12.382: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:12.382: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:44:12.382: INFO: pvc-volume-tester-mjlhh started at 2023-03-24 23:43:57 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:44:12.382: INFO: pod-disruption-failure-ignore-2-db75x started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: pod-disruption-failure-ignore-1-zdcbz started at 2023-03-24 23:44:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: fail-once-non-local-9wlqk started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: netserver-1 started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container webserver ready: false, restart count 0 Mar 24 23:44:12.382: INFO: pod-disruption-failure-ignore-3-jtnrz started at 2023-03-24 23:44:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:44:12.382: INFO: rs-94lz2 started at 2023-03-24 23:43:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container donothing ready: false, restart count 0 Mar 24 23:44:12.382: INFO: hostexec-i-06956c12f825d08d9-qfwqs started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:44:12.382: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:44:06 +0000 UTC (0+7 container statuses recorded) Mar 24 23:44:12.382: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:44:12.382: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:44:12.382: INFO: fail-once-non-local-kxf4m started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: dns-test-92bebb01-a737-48e0-8cbd-c482cf0492f2 started at 2023-03-24 23:43:48 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:12.382: INFO: Container jessie-querier ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container querier ready: true, restart count 0 Mar 24 23:44:12.382: INFO: Container webserver ready: true, restart count 0 Mar 24 23:44:12.382: INFO: pod-subpath-test-dynamicpv-wnb6 started at 2023-03-24 23:44:10 +0000 UTC (2+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Init container init-volume-dynamicpv-wnb6 ready: false, restart count 0 Mar 24 23:44:12.382: INFO: Init container test-init-volume-dynamicpv-wnb6 ready: false, restart count 0 Mar 24 23:44:12.382: INFO: Container test-container-subpath-dynamicpv-wnb6 ready: false, restart count 0 Mar 24 23:44:12.382: INFO: test-rolling-update-with-lb-69fcddc54f-42b69 started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:44:12.382: INFO: fail-once-non-local-ghrgt started at 2023-03-24 23:44:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container c ready: false, restart count 0 Mar 24 23:44:12.382: INFO: e2e-net-exec started at 2023-03-24 23:44:11 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.382: INFO: Container e2e-net-exec ready: false, restart count 0 Mar 24 23:44:12.731: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:44:12.731: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:44:12.794: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 32934 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:43:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:43:48 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:43:48 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:43:48 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:43:48 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:44:12.795: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:44:12.865: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:44:12.932: INFO: netserver-2 started at 2023-03-24 23:44:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container webserver ready: false, restart count 0 Mar 24 23:44:12.932: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container coredns ready: true, restart count 0 Mar 24 23:44:12.932: INFO: test-webserver-beecf91c-d4bc-4b66-9648-32e1c7cd7922 started at 2023-03-24 23:43:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container test-webserver ready: true, restart count 0 Mar 24 23:44:12.932: INFO: pod-secrets-be64fb13-7ee9-4ac0-a82e-6dd54f026006 started at 2023-03-24 23:44:11 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container secret-volume-test ready: false, restart count 0 Mar 24 23:44:12.932: INFO: condition-test-bs78d started at 2023-03-24 23:44:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container httpd ready: true, restart count 0 Mar 24 23:44:12.932: INFO: probe-test-6e7c4325-6673-424f-93fc-b3af435dc7d3 started at 2023-03-24 23:44:03 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container probe-test-6e7c4325-6673-424f-93fc-b3af435dc7d3 ready: true, restart count 0 Mar 24 23:44:12.932: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:44:12.932: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:44:12.932: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:44:12.932: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:44:12.932: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:12.932: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:44:12.932: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:12.932: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:44:12.932: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:12.932: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:44:13.181: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:44:13.181: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:44:13.244: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 22479 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:40:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:44:13.244: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:44:13.314: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:44:13.385: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:44:13.385: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:44:13.385: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:44:13.385: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:44:13.385: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:44:13.385: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:44:13.385: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:44:13.385: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:44:13.385: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:13.385: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:44:13.385: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:44:13.385: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:44:13.385: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:44:13.385: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:44:13.385: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.385: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:44:13.385: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:44:13.385: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:44:13.385: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:13.672: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:44:13.672: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:44:13.734: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 33673 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-expansion-3855":"i-0cdea7500a633a81f","csi-mock-csi-mock-volumes-workload-6274":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:44:10 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:44:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:10 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:10 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:10 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:44:10 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b,DevicePath:,},},Config:nil,},} Mar 24 23:44:13.735: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:44:13.804: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:44:13.876: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:44:13.876: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:44:13.876: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container coredns ready: true, restart count 0 Mar 24 23:44:13.876: INFO: netserver-3 started at 2023-03-24 23:44:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container webserver ready: false, restart count 0 Mar 24 23:44:13.876: INFO: e2e-net-server started at 2023-03-24 23:44:13 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container e2e-net-server ready: false, restart count 0 Mar 24 23:44:13.876: INFO: csi-mockplugin-0 started at 2023-03-24 23:38:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:13.876: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container mock ready: true, restart count 0 Mar 24 23:44:13.876: INFO: pvc-volume-tester-vgkcq started at 2023-03-24 23:38:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:44:13.876: INFO: condition-test-n9t4m started at 2023-03-24 23:44:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container httpd ready: true, restart count 0 Mar 24 23:44:13.876: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:44:13.876: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:44:13.876: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:41:48 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:44:13.876: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:13.876: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:44:13.876: INFO: csi-mockplugin-0 started at 2023-03-24 23:41:48 +0000 UTC (0+3 container statuses recorded) Mar 24 23:44:13.876: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:44:13.876: INFO: Container mock ready: true, restart count 0 Mar 24 23:44:13.876: INFO: test-rolling-update-with-lb-69fcddc54f-2hq6t started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:44:13.876: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:44:14.343: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:44:14.343 (2.883s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:44:14.343 (2.883s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:44:14.343 STEP: Destroying namespace "provisioning-2859" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:44:14.343 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:44:14.409 (66ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:44:14.409 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:44:14.409 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sexisting\ssingle\sfile\s\[LinuxOnly\]$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-6zsv for test-container-subpath-dynamicpv-6zsv: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8350/pods/pod-subpath-test-dynamicpv-6zsv/log?container=test-container-subpath-dynamicpv-6zsv&previous=false": stream error: stream ID 2467; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:44:47.352from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:44:29.746 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:44:29.746 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:44:29.746 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:44:29.746 Mar 24 23:44:29.746: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:44:29.747 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:44:29.939 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:44:30.068 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:44:30.194 (448ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:44:30.194 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:44:30.194 (0s) > Enter [It] should support existing single file [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:219 @ 03/24/23 23:44:30.194 Mar 24 23:44:30.194: INFO: Creating resource for dynamic PV Mar 24 23:44:30.194: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-8350-e2e-scsvgvs - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:44:30.194 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:44:30.259 Mar 24 23:44:30.259: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-6zsv - test/e2e/storage/testsuites/subpath.go:691 @ 03/24/23 23:44:30.388 STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:44:30.388 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:44:47.091 Mar 24 23:44:47.154: INFO: Trying to get logs from node i-0713c0b421302f422 pod pod-subpath-test-dynamicpv-6zsv container test-container-subpath-dynamicpv-6zsv: <nil> Mar 24 23:44:47.217: INFO: Failed to get logs from node "i-0713c0b421302f422" pod "pod-subpath-test-dynamicpv-6zsv" container "test-container-subpath-dynamicpv-6zsv". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8350/pods/pod-subpath-test-dynamicpv-6zsv/log?container=test-container-subpath-dynamicpv-6zsv&previous=false": stream error: stream ID 2467; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:44:47.217 Mar 24 23:44:47.352: INFO: Unexpected error: <*fmt.wrapError | 0xc0010bb400>: failed to get logs from pod-subpath-test-dynamicpv-6zsv for test-container-subpath-dynamicpv-6zsv: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8350/pods/pod-subpath-test-dynamicpv-6zsv/log?container=test-container-subpath-dynamicpv-6zsv&previous=false": stream error: stream ID 2467; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-6zsv for test-container-subpath-dynamicpv-6zsv: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8350/pods/pod-subpath-test-dynamicpv-6zsv/log?container=test-container-subpath-dynamicpv-6zsv&previous=false\": stream error: stream ID 2467; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc0029cc750>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8350/pods/pod-subpath-test-dynamicpv-6zsv/log?container=test-container-subpath-dynamicpv-6zsv&previous=false", Err: <http2.StreamError>{ StreamID: 2467, Code: 2, Cause: <*errors.errorString | 0xc0001b3650>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-6zsv for test-container-subpath-dynamicpv-6zsv: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8350/pods/pod-subpath-test-dynamicpv-6zsv/log?container=test-container-subpath-dynamicpv-6zsv&previous=false": stream error: stream ID 2467; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:44:47.352 < Exit [It] should support existing single file [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:219 @ 03/24/23 23:44:47.352 (17.159s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:44:47.352 Mar 24 23:44:47.352: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:44:47.417 (65ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:221 @ 03/24/23 23:44:47.417 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:44:47.418 Mar 24 23:44:47.418: INFO: Deleting pod "pod-subpath-test-dynamicpv-6zsv" in namespace "provisioning-8350" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:44:47.483 Mar 24 23:44:47.612: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com2xf9p" Mar 24 23:44:47.677: INFO: Waiting up to 5m0s for PersistentVolume pvc-64054c37-35b1-4932-acea-679e15ffec45 to get deleted Mar 24 23:44:47.741: INFO: PersistentVolume pvc-64054c37-35b1-4932-acea-679e15ffec45 found and phase=Released (63.106117ms) Mar 24 23:44:52.809: INFO: PersistentVolume pvc-64054c37-35b1-4932-acea-679e15ffec45 found and phase=Released (5.131076812s) Mar 24 23:44:57.872: INFO: PersistentVolume pvc-64054c37-35b1-4932-acea-679e15ffec45 found and phase=Released (10.194969943s) Mar 24 23:45:02.937: INFO: PersistentVolume pvc-64054c37-35b1-4932-acea-679e15ffec45 was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:45:02.937 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:221 @ 03/24/23 23:45:03.002 (15.585s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:45:03.002 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:45:03.002 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:45:03.002 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:45:03.002 STEP: Collecting events from namespace "provisioning-8350". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:45:03.002 STEP: Found 12 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:45:03.067 Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:30 +0000 UTC - event for ebs.csi.aws.com2xf9p: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:30 +0000 UTC - event for ebs.csi.aws.com2xf9p: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-8350/ebs.csi.aws.com2xf9p" Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:30 +0000 UTC - event for ebs.csi.aws.com2xf9p: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:33 +0000 UTC - event for ebs.csi.aws.com2xf9p: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-64054c37-35b1-4932-acea-679e15ffec45 Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:34 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {default-scheduler } Scheduled: Successfully assigned provisioning-8350/pod-subpath-test-dynamicpv-6zsv to i-0713c0b421302f422 Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:36 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-64054c37-35b1-4932-acea-679e15ffec45" Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:40 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:40 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {kubelet i-0713c0b421302f422} Created: Created container init-volume-dynamicpv-6zsv Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:41 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {kubelet i-0713c0b421302f422} Started: Started container init-volume-dynamicpv-6zsv Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:42 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:42 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {kubelet i-0713c0b421302f422} Created: Created container test-container-subpath-dynamicpv-6zsv Mar 24 23:45:03.067: INFO: At 2023-03-24 23:44:42 +0000 UTC - event for pod-subpath-test-dynamicpv-6zsv: {kubelet i-0713c0b421302f422} Started: Started container test-container-subpath-dynamicpv-6zsv Mar 24 23:45:03.130: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:45:03.130: INFO: Mar 24 23:45:03.195: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:45:03.264: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 35958 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-capacity-6575":"i-005794773d640afd9","csi-mock-csi-mock-volumes-expansion-4404":"csi-mock-csi-mock-volumes-expansion-4404","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:44:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:44:59 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:28 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:28 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:28 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:44:28 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-4404^4,DevicePath:,},},Config:nil,},} Mar 24 23:45:03.264: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:45:03.342: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:45:03.416: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:45:03.416: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:44:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:45:03.416: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:45:03.416: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:45:03.416: INFO: csi-mockplugin-0 started at 2023-03-24 23:44:46 +0000 UTC (0+4 container statuses recorded) Mar 24 23:45:03.416: INFO: Container busybox ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container mock ready: true, restart count 0 Mar 24 23:45:03.416: INFO: csi-mockplugin-0 started at 2023-03-24 23:42:03 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:03.416: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container mock ready: true, restart count 0 Mar 24 23:45:03.416: INFO: boom-server started at 2023-03-24 23:44:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container boom-server ready: true, restart count 0 Mar 24 23:45:03.416: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:44:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:45:03.416: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:03.416: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:45:03.416: INFO: ss-2 started at 2023-03-24 23:45:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:03.416: INFO: liveness-92f7bad4-2180-4189-9432-5763c759fd4f started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:45:03.416: INFO: netserver-0 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:03.416: INFO: csi-mockplugin-0 started at 2023-03-24 23:43:49 +0000 UTC (0+4 container statuses recorded) Mar 24 23:45:03.416: INFO: Container busybox ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:45:03.416: INFO: Container mock ready: true, restart count 0 Mar 24 23:45:03.416: INFO: pvc-volume-tester-s9ntj started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:45:03.416: INFO: test-rolling-update-with-lb-66756c6d5c-hxtp2 started at 2023-03-24 23:44:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:45:03.416: INFO: netserver-0 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:03.416: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:04.023: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:45:04.023: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:45:04.087: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 35995 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-4389":"i-06956c12f825d08d9","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:45:00 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:45:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:45:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0cd8d340169ee4b11],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0cd8d340169ee4b11,DevicePath:,},},Config:nil,},} Mar 24 23:45:04.087: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:45:04.159: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:45:04.230: INFO: pfpod started at 2023-03-24 23:43:25 +0000 UTC (0+2 container statuses recorded) Mar 24 23:45:04.230: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:45:04.230: INFO: Container readiness ready: false, restart count 0 Mar 24 23:45:04.230: INFO: pod-secrets-89f39212-2f92-452d-a143-8ea5b1f722b0 started at 2023-03-24 23:43:46 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:04.230: INFO: Container creates-volume-test ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container dels-volume-test ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container upds-volume-test ready: true, restart count 0 Mar 24 23:45:04.230: INFO: startup-script started at 2023-03-24 23:44:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container startup-script ready: true, restart count 0 Mar 24 23:45:04.230: INFO: pod-ephm-test-configmap-gscq started at 2023-03-24 23:43:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container test-container-subpath-configmap-gscq ready: false, restart count 0 Mar 24 23:45:04.230: INFO: test-rolling-update-with-lb-66756c6d5c-5nbqq started at 2023-03-24 23:44:48 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:45:04.230: INFO: pfpod started at 2023-03-24 23:44:20 +0000 UTC (0+2 container statuses recorded) Mar 24 23:45:04.230: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:45:04.230: INFO: Container readiness ready: false, restart count 0 Mar 24 23:45:04.230: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:04.230: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:45:04.230: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:44:40 +0000 UTC (0+7 container statuses recorded) Mar 24 23:45:04.230: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:45:04.230: INFO: netserver-1 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:04.230: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:45:04.230: INFO: rs-94lz2 started at 2023-03-24 23:43:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container donothing ready: false, restart count 0 Mar 24 23:45:04.230: INFO: netserver-1 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:04.230: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:45:04.230: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:45:04.230: INFO: ss-1 started at 2023-03-24 23:44:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.230: INFO: Container webserver ready: true, restart count 0 Mar 24 23:45:04.738: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:45:04.738: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:45:04.802: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 35507 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-fsgroup-policy-6281":"i-0713c0b421302f422","ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:44:49 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:49 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:49 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:44:49 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:44:49 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-088904af6e435f33b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-088904af6e435f33b,DevicePath:,},},Config:nil,},} Mar 24 23:45:04.802: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:45:04.882: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:45:04.955: INFO: netserver-2 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:04.955: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:45:04.955: INFO: pfpod started at 2023-03-24 23:44:53 +0000 UTC (0+2 container statuses recorded) Mar 24 23:45:04.955: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:45:04.955: INFO: Container readiness ready: true, restart count 0 Mar 24 23:45:04.955: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:45:04.955: INFO: execpod5tlwv started at 2023-03-24 23:44:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:45:04.955: INFO: pvc-volume-tester-qmskm started at 2023-03-24 23:44:53 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:45:04.955: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:04.955: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:45:04.955: INFO: netserver-2 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:04.955: INFO: ss-0 started at 2023-03-24 23:44:20 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container webserver ready: true, restart count 0 Mar 24 23:45:04.955: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container coredns ready: true, restart count 0 Mar 24 23:45:04.955: INFO: nodeport-update-service-tz54p started at 2023-03-24 23:44:49 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 24 23:45:04.955: INFO: test-webserver-beecf91c-d4bc-4b66-9648-32e1c7cd7922 started at 2023-03-24 23:43:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container test-webserver ready: true, restart count 0 Mar 24 23:45:04.955: INFO: liveness-f8974ac8-e336-4e86-bf14-c957462aa620 started at 2023-03-24 23:44:56 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:04.955: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:45:04.955: INFO: csi-mockplugin-0 started at 2023-03-24 23:44:42 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:04.955: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:45:04.955: INFO: Container mock ready: true, restart count 0 Mar 24 23:45:05.256: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:45:05.256: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:45:05.320: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 22479 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:40:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:45:05.320: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:45:05.392: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:45:05.463: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:45:05.463: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:45:05.463: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:45:05.463: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:45:05.463: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:45:05.463: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:45:05.463: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:45:05.463: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:05.463: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:45:05.463: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:45:05.463: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:45:05.463: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:05.463: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:45:05.463: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:45:05.463: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:45:05.463: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:45:05.463: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:45:05.463: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.463: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:45:05.760: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:45:05.760: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:45:05.829: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 36033 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-expansion-3855":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:45:02 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:45:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:02 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:02 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:02 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:45:02 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b,DevicePath:,},},Config:nil,},} Mar 24 23:45:05.830: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:45:05.907: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:45:05.978: INFO: netserver-3 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:05.978: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:45:05.978: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:45:05.978: INFO: netserver-3 started at 2023-03-24 23:44:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container webserver ready: false, restart count 0 Mar 24 23:45:05.978: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:05.978: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:45:05.978: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:45:05.978: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:45:05.978: INFO: hostexec-i-0cdea7500a633a81f-zx8r2 started at 2023-03-24 23:44:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:45:05.978: INFO: test-rolling-update-with-lb-66756c6d5c-rg4zh started at 2023-03-24 23:44:45 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:45:05.978: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:45:05.978: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:45:05.978: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:45:05.978: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:45:05.978: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container coredns ready: true, restart count 0 Mar 24 23:45:05.978: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.978: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:45:05.978: INFO: csi-mockplugin-0 started at 2023-03-24 23:38:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:45:05.979: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:45:05.979: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:45:05.979: INFO: Container mock ready: true, restart count 0 Mar 24 23:45:05.979: INFO: pvc-volume-tester-vgkcq started at 2023-03-24 23:38:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.979: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:45:05.979: INFO: nodeport-update-service-lmf82 started at 2023-03-24 23:44:49 +0000 UTC (0+1 container statuses recorded) Mar 24 23:45:05.979: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 24 23:45:06.308: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:45:06.308 (3.306s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:45:06.308 (3.306s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:45:06.308 STEP: Destroying namespace "provisioning-8350" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:45:06.308 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:45:06.385 (77ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:45:06.385 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:45:06.385 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sfile\sas\ssubpath\s\[LinuxOnly\]$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-jmfk for test-container-subpath-dynamicpv-jmfk: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-3289/pods/pod-subpath-test-dynamicpv-jmfk/log?container=test-container-subpath-dynamicpv-jmfk&previous=false": stream error: stream ID 111; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:48:20.071from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:47:41.868 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:47:41.868 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:47:41.868 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:47:41.868 Mar 24 23:47:41.868: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:47:41.87 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:47:42.057 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:47:42.18 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:47:42.308 (440ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:47:42.308 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:47:42.308 (0s) > Enter [It] should support file as subpath [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:230 @ 03/24/23 23:47:42.308 Mar 24 23:47:42.308: INFO: Creating resource for dynamic PV Mar 24 23:47:42.308: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-3289-e2e-sclwpbf - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:47:42.309 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:47:42.374 Mar 24 23:47:42.374: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-jmfk - test/e2e/storage/testsuites/subpath.go:489 @ 03/24/23 23:47:42.51 STEP: Creating a pod to test atomic-volume-subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:47:42.51 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:48:19.81 Mar 24 23:48:19.872: INFO: Trying to get logs from node i-0713c0b421302f422 pod pod-subpath-test-dynamicpv-jmfk container test-container-subpath-dynamicpv-jmfk: <nil> Mar 24 23:48:19.934: INFO: Failed to get logs from node "i-0713c0b421302f422" pod "pod-subpath-test-dynamicpv-jmfk" container "test-container-subpath-dynamicpv-jmfk". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-3289/pods/pod-subpath-test-dynamicpv-jmfk/log?container=test-container-subpath-dynamicpv-jmfk&previous=false": stream error: stream ID 111; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:48:19.934 Mar 24 23:48:20.071: INFO: Unexpected error: <*fmt.wrapError | 0xc008f35f60>: failed to get logs from pod-subpath-test-dynamicpv-jmfk for test-container-subpath-dynamicpv-jmfk: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-3289/pods/pod-subpath-test-dynamicpv-jmfk/log?container=test-container-subpath-dynamicpv-jmfk&previous=false": stream error: stream ID 111; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-jmfk for test-container-subpath-dynamicpv-jmfk: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-3289/pods/pod-subpath-test-dynamicpv-jmfk/log?container=test-container-subpath-dynamicpv-jmfk&previous=false\": stream error: stream ID 111; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc001767380>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-3289/pods/pod-subpath-test-dynamicpv-jmfk/log?container=test-container-subpath-dynamicpv-jmfk&previous=false", Err: <http2.StreamError>{ StreamID: 111, Code: 2, Cause: <*errors.errorString | 0xc0001b3670>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-jmfk for test-container-subpath-dynamicpv-jmfk: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-3289/pods/pod-subpath-test-dynamicpv-jmfk/log?container=test-container-subpath-dynamicpv-jmfk&previous=false": stream error: stream ID 111; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:48:20.071 < Exit [It] should support file as subpath [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:230 @ 03/24/23 23:48:20.071 (37.763s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:48:20.071 Mar 24 23:48:20.071: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:48:20.135 (64ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:232 @ 03/24/23 23:48:20.135 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:48:20.136 Mar 24 23:48:20.136: INFO: Deleting pod "pod-subpath-test-dynamicpv-jmfk" in namespace "provisioning-3289" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:48:20.197 Mar 24 23:48:20.324: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comgsbpt" Mar 24 23:48:20.388: INFO: Waiting up to 5m0s for PersistentVolume pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2 to get deleted Mar 24 23:48:20.450: INFO: PersistentVolume pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2 found and phase=Released (61.515647ms) Mar 24 23:48:25.513: INFO: PersistentVolume pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2 found and phase=Released (5.124483918s) Mar 24 23:48:30.597: INFO: PersistentVolume pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2 found and phase=Released (10.208442362s) Mar 24 23:48:35.659: INFO: PersistentVolume pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2 was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:48:35.659 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:232 @ 03/24/23 23:48:35.723 (15.587s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:48:35.723 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:48:35.723 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:48:35.723 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:48:35.723 STEP: Collecting events from namespace "provisioning-3289". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:48:35.723 STEP: Found 12 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:48:35.785 Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:42 +0000 UTC - event for ebs.csi.aws.comgsbpt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:42 +0000 UTC - event for ebs.csi.aws.comgsbpt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:42 +0000 UTC - event for ebs.csi.aws.comgsbpt: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-3289/ebs.csi.aws.comgsbpt" Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:45 +0000 UTC - event for ebs.csi.aws.comgsbpt: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2 Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:46 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {default-scheduler } Scheduled: Successfully assigned provisioning-3289/pod-subpath-test-dynamicpv-jmfk to i-0713c0b421302f422 Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:48 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-4ecba2dd-463e-436c-91d9-ab34a58ce7b2" Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:54 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:54 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {kubelet i-0713c0b421302f422} Created: Created container init-volume-dynamicpv-jmfk Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:54 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {kubelet i-0713c0b421302f422} Started: Started container init-volume-dynamicpv-jmfk Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:55 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:55 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {kubelet i-0713c0b421302f422} Created: Created container test-container-subpath-dynamicpv-jmfk Mar 24 23:48:35.785: INFO: At 2023-03-24 23:47:55 +0000 UTC - event for pod-subpath-test-dynamicpv-jmfk: {kubelet i-0713c0b421302f422} Started: Started container test-container-subpath-dynamicpv-jmfk Mar 24 23:48:35.853: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:48:35.853: INFO: Mar 24 23:48:35.915: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:48:35.977: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 46605 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4698":"i-005794773d640afd9","csi-hostpath-ephemeral-7616":"i-005794773d640afd9","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:48:22 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:48:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:48:22 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:48:22 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:48:22 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:48:22 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-4698^51c7b929-ca9e-11ed-9ea6-42cc3bbf9ce2 kubernetes.io/csi/csi-hostpath-ephemeral-4698^51cd5a91-ca9e-11ed-9ea6-42cc3bbf9ce2],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-4698^51c7b929-ca9e-11ed-9ea6-42cc3bbf9ce2,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-4698^51cd5a91-ca9e-11ed-9ea6-42cc3bbf9ce2,DevicePath:,},},Config:nil,},} Mar 24 23:48:35.977: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:48:36.047: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:48:36.179: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:48:36.179: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:48:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:48:36.179: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:48:36.179: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:48:36.179: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:48:02 +0000 UTC (0+7 container statuses recorded) Mar 24 23:48:36.179: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-7b75d79cf5-6g6qq started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-67bd4bf6dc-jhzmm started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.179: INFO: annotationupdated6d1ee1a-7c07-45ab-8836-dd4eb54be843 started at 2023-03-24 23:47:28 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container client-container ready: true, restart count 0 Mar 24 23:48:36.179: INFO: csi-mockplugin-0 started at 2023-03-24 23:48:29 +0000 UTC (0+4 container statuses recorded) Mar 24 23:48:36.179: INFO: Container busybox ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container mock ready: true, restart count 0 Mar 24 23:48:36.179: INFO: pod-projected-configmaps-cd5dc486-c3e2-4aed-b2e5-a109d691d2a4 started at 2023-03-24 23:47:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-7b75d79cf5-bzhrh started at 2023-03-24 23:48:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-67bd4bf6dc-4vz4z started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.179: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:48:36.179: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:36.179: INFO: annotationupdate1566e306-b227-4b68-80c2-f0b730a497a8 started at 2023-03-24 23:47:41 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container client-container ready: true, restart count 0 Mar 24 23:48:36.179: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:42 +0000 UTC (0+7 container statuses recorded) Mar 24 23:48:36.179: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-67bd4bf6dc-5zpxr started at 2023-03-24 23:48:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.179: INFO: pod-with-poststart-http-hook started at 2023-03-24 23:47:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container pod-with-poststart-http-hook ready: true, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-7b75d79cf5-k9dnl started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-67bd4bf6dc-787tz started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.179: INFO: inline-volume-tester-blzbc started at 2023-03-24 23:47:42 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:48:36.179: INFO: pod-handle-http-request started at 2023-03-24 23:47:13 +0000 UTC (0+2 container statuses recorded) Mar 24 23:48:36.179: INFO: Container container-handle-http-request ready: true, restart count 0 Mar 24 23:48:36.179: INFO: Container container-handle-https-request ready: true, restart count 0 Mar 24 23:48:36.179: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:56 +0000 UTC (0+7 container statuses recorded) Mar 24 23:48:36.179: INFO: Container csi-attacher ready: false, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-provisioner ready: false, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-resizer ready: false, restart count 0 Mar 24 23:48:36.179: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 24 23:48:36.179: INFO: Container hostpath ready: false, restart count 0 Mar 24 23:48:36.179: INFO: Container liveness-probe ready: false, restart count 0 Mar 24 23:48:36.179: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 24 23:48:36.179: INFO: inline-volume-tester-dx2bf started at 2023-03-24 23:48:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:48:36.179: INFO: webserver-deployment-67bd4bf6dc-njm7m started at 2023-03-24 23:48:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.179: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.585: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:48:36.585: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:48:36.654: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 46544 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:47:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:48:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-05626d827e3c019cb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-05626d827e3c019cb,DevicePath:,},},Config:nil,},} Mar 24 23:48:36.654: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:48:36.742: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:48:36.815: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:48:36.815: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:48:36.815: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:48:36.815: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:48:36.815: INFO: suspend-false-to-true-56xcb started at 2023-03-24 23:48:31 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container c ready: true, restart count 0 Mar 24 23:48:36.815: INFO: inline-volume-tester-wpdjn started at 2023-03-24 23:47:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:48:36.815: INFO: webserver-deployment-67bd4bf6dc-xr2sl started at 2023-03-24 23:48:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.815: INFO: webserver-deployment-7b75d79cf5-qssws started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.815: INFO: webserver-deployment-7b75d79cf5-7kt8k started at 2023-03-24 23:48:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.815: INFO: busybox-readonly-fs66f74748-9b83-46c5-8cb3-f24acf5a3af3 started at 2023-03-24 23:48:20 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container busybox-readonly-fs66f74748-9b83-46c5-8cb3-f24acf5a3af3 ready: true, restart count 0 Mar 24 23:48:36.815: INFO: webserver-deployment-7b75d79cf5-jj8vk started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.815: INFO: webserver-deployment-7b75d79cf5-7njc8 started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container httpd ready: false, restart count 0 Mar 24 23:48:36.815: INFO: downwardapi-volume-fd31d8a5-310e-472d-9bfa-6c29170fe8f4 started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container client-container ready: false, restart count 0 Mar 24 23:48:36.815: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:48:36.815: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:48:36.815: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:36.815: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:36.815: INFO: e2e-zh847-2mnz7 started at 2023-03-24 23:48:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.815: INFO: Container c ready: false, restart count 0 Mar 24 23:48:36.815: INFO: webserver-deployment-67bd4bf6dc-5nblp started at 2023-03-24 23:48:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.816: INFO: Container httpd ready: true, restart count 0 Mar 24 23:48:36.816: INFO: suspend-false-to-true-wxjsl started at 2023-03-24 23:48:31 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.816: INFO: Container c ready: true, restart count 0 Mar 24 23:48:36.816: INFO: e2e-zh847-trk4q started at 2023-03-24 23:48:33 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.816: INFO: Container c ready: false, restart count 0 Mar 24 23:48:36.816: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.816: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:48:36.816: INFO: e2e-zh847-s4zs9 started at 2023-03-24 23:48:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:36.816: INFO: Container c ready: false, restart count 0 Mar 24 23:48:37.423: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:48:37.423: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:48:37.485: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 45147 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3735":"i-0713c0b421302f422","csi-hostpath-ephemeral-6856":"i-0713c0b421302f422","ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:48:24 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:48:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:48:24 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:48:24 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:48:24 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:48:24 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3735^3c64aba5-ca9e-11ed-934b-76555be6a59a kubernetes.io/csi/csi-hostpath-ephemeral-6856^4f9f6315-ca9e-11ed-9d61-a248529a3895 kubernetes.io/csi/csi-hostpath-ephemeral-6856^4f9f7a9d-ca9e-11ed-9d61-a248529a3895],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6856^4f9f7a9d-ca9e-11ed-9d61-a248529a3895,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3735^3c64aba5-ca9e-11ed-934b-76555be6a59a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6856^4f9f6315-ca9e-11ed-9d61-a248529a3895,DevicePath:,},},Config:nil,},} Mar 24 23:48:37.485: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:48:37.574: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:48:37.650: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:48:37.650: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:48:37.650: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:37.650: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:48:37.650: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:48:37.650: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:37.650: INFO: inline-volume-tester-2fgw7 started at 2023-03-24 23:47:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:37.650: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:48:37.650: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:37.650: INFO: Container coredns ready: true, restart count 0 Mar 24 23:48:37.650: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:58 +0000 UTC (0+7 container statuses recorded) Mar 24 23:48:37.650: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:37.650: INFO: pfpod started at 2023-03-24 23:47:00 +0000 UTC (0+2 container statuses recorded) Mar 24 23:48:37.650: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:48:37.650: INFO: Container readiness ready: false, restart count 0 Mar 24 23:48:37.650: INFO: inline-volume-tester-75hf8 started at 2023-03-24 23:48:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:37.650: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:48:37.650: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:26 +0000 UTC (0+7 container statuses recorded) Mar 24 23:48:37.650: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:37.650: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:37.650: INFO: startup-da1780a0-61c5-4110-8f42-ae25c8c37b21 started at 2023-03-24 23:47:44 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:37.650: INFO: Container busybox ready: true, restart count 0 Mar 24 23:48:38.088: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:48:38.088: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:48:38.211: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 36523 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:45:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:48:38.212: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:48:38.327: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:48:38.402: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:48:38.402: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:48:38.402: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:48:38.402: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:48:38.402: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:48:38.402: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:48:38.402: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:48:38.402: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:48:38.402: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:48:38.402: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:48:38.402: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:48:38.402: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:48:38.402: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:48:38.402: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:48:38.402: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:48:38.402: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:38.402: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:48:38.402: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:48:38.402: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:38.765: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:48:38.765: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:48:38.834: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 42964 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:47:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:48:38.835: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:48:38.934: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:48:39.029: INFO: pod-configmaps-4faa40ec-e6e6-4f18-a7f9-fbad3cc24f7e started at 2023-03-24 23:46:35 +0000 UTC (0+3 container statuses recorded) Mar 24 23:48:39.029: INFO: Container createcm-volume-test ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Container delcm-volume-test ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Container updcm-volume-test ready: true, restart count 0 Mar 24 23:48:39.029: INFO: labelsupdate1991bed5-3523-4380-9fbc-c19af67dc428 started at 2023-03-24 23:48:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Container client-container ready: true, restart count 0 Mar 24 23:48:39.029: INFO: e2e-zh847-99pnw started at 2023-03-24 23:48:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Container c ready: false, restart count 0 Mar 24 23:48:39.029: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:48:39.029: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:48:39.029: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:48:39.029: INFO: concurrent-27995028-66r2k started at 2023-03-24 23:48:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Container c ready: true, restart count 0 Mar 24 23:48:39.029: INFO: pause-pod-1 started at 2023-03-24 23:48:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:48:39.029: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:48:39.029: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:48:39.029: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:48:39.029: INFO: Container coredns ready: true, restart count 0 Mar 24 23:48:39.342: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:48:39.342 (3.619s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:48:39.342 (3.619s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:48:39.342 STEP: Destroying namespace "provisioning-3289" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:48:39.342 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:48:39.448 (106ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:48:39.448 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:48:39.448 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\snon\-existent\spath$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-752j for test-container-volume-dynamicpv-752j: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-6035/pods/pod-subpath-test-dynamicpv-752j/log?container=test-container-volume-dynamicpv-752j&previous=false": stream error: stream ID 1055; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:42:35.886from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:42:16.15 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:42:16.15 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:42:16.15 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:42:16.151 Mar 24 23:42:16.151: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:42:16.151 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:42:16.347 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:42:16.474 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:42:16.602 (452ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:42:16.602 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:42:16.602 (0s) > Enter [It] should support non-existent path - test/e2e/storage/testsuites/subpath.go:194 @ 03/24/23 23:42:16.602 Mar 24 23:42:16.602: INFO: Creating resource for dynamic PV Mar 24 23:42:16.602: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-6035-e2e-scvqpf9 - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:42:16.603 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:42:16.667 Mar 24 23:42:16.667: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-752j - test/e2e/storage/testsuites/subpath.go:691 @ 03/24/23 23:42:16.799 STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:42:16.799 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:42:35.524 Mar 24 23:42:35.621: INFO: Trying to get logs from node i-06956c12f825d08d9 pod pod-subpath-test-dynamicpv-752j container test-container-volume-dynamicpv-752j: <nil> Mar 24 23:42:35.698: INFO: Failed to get logs from node "i-06956c12f825d08d9" pod "pod-subpath-test-dynamicpv-752j" container "test-container-volume-dynamicpv-752j". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-6035/pods/pod-subpath-test-dynamicpv-752j/log?container=test-container-volume-dynamicpv-752j&previous=false": stream error: stream ID 1055; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:42:35.698 Mar 24 23:42:35.886: INFO: Unexpected error: <*fmt.wrapError | 0xc005164a60>: failed to get logs from pod-subpath-test-dynamicpv-752j for test-container-volume-dynamicpv-752j: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-6035/pods/pod-subpath-test-dynamicpv-752j/log?container=test-container-volume-dynamicpv-752j&previous=false": stream error: stream ID 1055; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-752j for test-container-volume-dynamicpv-752j: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-6035/pods/pod-subpath-test-dynamicpv-752j/log?container=test-container-volume-dynamicpv-752j&previous=false\": stream error: stream ID 1055; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc005639aa0>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-6035/pods/pod-subpath-test-dynamicpv-752j/log?container=test-container-volume-dynamicpv-752j&previous=false", Err: <http2.StreamError>{ StreamID: 1055, Code: 2, Cause: <*errors.errorString | 0xc0001b3690>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-752j for test-container-volume-dynamicpv-752j: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-6035/pods/pod-subpath-test-dynamicpv-752j/log?container=test-container-volume-dynamicpv-752j&previous=false": stream error: stream ID 1055; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:42:35.886 < Exit [It] should support non-existent path - test/e2e/storage/testsuites/subpath.go:194 @ 03/24/23 23:42:35.886 (19.284s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:42:35.886 Mar 24 23:42:35.886: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:42:35.963 (77ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:196 @ 03/24/23 23:42:35.963 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:42:35.963 Mar 24 23:42:35.963: INFO: Deleting pod "pod-subpath-test-dynamicpv-752j" in namespace "provisioning-6035" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:42:36.039 Mar 24 23:42:36.180: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comjpft6" Mar 24 23:42:36.251: INFO: Waiting up to 5m0s for PersistentVolume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf to get deleted Mar 24 23:42:36.321: INFO: PersistentVolume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf found and phase=Released (70.401396ms) Mar 24 23:42:41.394: INFO: PersistentVolume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf found and phase=Released (5.143725878s) Mar 24 23:42:46.460: INFO: PersistentVolume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf found and phase=Released (10.208784566s) Mar 24 23:42:51.529: INFO: PersistentVolume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf found and phase=Released (15.278692472s) Mar 24 23:42:56.594: INFO: PersistentVolume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:42:56.594 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:196 @ 03/24/23 23:42:56.663 (20.7s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:42:56.663 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:42:56.663 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:42:56.663 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:42:56.663 STEP: Collecting events from namespace "provisioning-6035". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:42:56.663 STEP: Found 15 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:42:56.73 Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:16 +0000 UTC - event for ebs.csi.aws.comjpft6: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:16 +0000 UTC - event for ebs.csi.aws.comjpft6: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:16 +0000 UTC - event for ebs.csi.aws.comjpft6: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-6035/ebs.csi.aws.comjpft6" Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:20 +0000 UTC - event for ebs.csi.aws.comjpft6: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:20 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {default-scheduler } Scheduled: Successfully assigned provisioning-6035/pod-subpath-test-dynamicpv-752j to i-06956c12f825d08d9 Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:23 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-3acb3f8c-7041-4da6-b697-5c324ebeecaf" Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:30 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:30 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Created: Created container test-init-subpath-dynamicpv-752j Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:30 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Started: Started container test-init-subpath-dynamicpv-752j Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:31 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:31 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Created: Created container test-container-subpath-dynamicpv-752j Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:31 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Started: Started container test-container-subpath-dynamicpv-752j Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:31 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:31 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Created: Created container test-container-volume-dynamicpv-752j Mar 24 23:42:56.730: INFO: At 2023-03-24 23:42:31 +0000 UTC - event for pod-subpath-test-dynamicpv-752j: {kubelet i-06956c12f825d08d9} Started: Started container test-container-volume-dynamicpv-752j Mar 24 23:42:56.798: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:42:56.798: INFO: Mar 24 23:42:56.868: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:42:56.993: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 27717 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-capacity-6575":"i-005794773d640afd9","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:41:56 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:42:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:15 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:15 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:15 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:42:15 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-01e96e7770b90bea6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-01e96e7770b90bea6,DevicePath:,},},Config:nil,},} Mar 24 23:42:56.993: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:42:57.078: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:42:57.151: INFO: csi-mockplugin-0 started at 2023-03-24 23:42:03 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:57.151: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Container mock ready: true, restart count 0 Mar 24 23:42:57.151: INFO: simpletest-rc-to-be-deleted-ht9jk started at 2023-03-24 23:42:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container nginx ready: true, restart count 0 Mar 24 23:42:57.151: INFO: ss2-1 started at 2023-03-24 23:42:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container webserver ready: true, restart count 0 Mar 24 23:42:57.151: INFO: simpletest-rc-to-be-deleted-g4skc started at 2023-03-24 23:42:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container nginx ready: true, restart count 0 Mar 24 23:42:57.151: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:57.151: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:57.151: INFO: pod-projected-configmaps-0041e20e-61be-4ff2-b71e-068a61cde1d8 started at 2023-03-24 23:38:48 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:57.151: INFO: Container createcm-volume-test ready: false, restart count 0 Mar 24 23:42:57.151: INFO: Container delcm-volume-test ready: false, restart count 0 Mar 24 23:42:57.151: INFO: Container updcm-volume-test ready: false, restart count 0 Mar 24 23:42:57.151: INFO: liveness-92f7bad4-2180-4189-9432-5763c759fd4f started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:42:57.151: INFO: verify-service-up-exec-pod-75ldx started at 2023-03-24 23:42:56 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container agnhost-container ready: false, restart count 0 Mar 24 23:42:57.151: INFO: up-down-3-t87nt started at 2023-03-24 23:42:51 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container up-down-3 ready: true, restart count 0 Mar 24 23:42:57.151: INFO: inline-volume-tester-5wwss started at 2023-03-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:42:57.151: INFO: test-rolling-update-with-lb-69fcddc54f-6n69z started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:42:57.151: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:42:57.151: INFO: up-down-2-2sjcs started at 2023-03-24 23:42:09 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container up-down-2 ready: true, restart count 0 Mar 24 23:42:57.151: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:42:57.151: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:42:57.151: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.151: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:42:57.506: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:42:57.506: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:42:57.581: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 29641 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6480":"i-06956c12f825d08d9","csi-mock-csi-mock-volumes-expansion-7358":"csi-mock-csi-mock-volumes-expansion-7358","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:42:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:42:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:47 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:47 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:47 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:42:47 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6480^8b488b43-ca9d-11ed-948f-b6de23070918 kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-7358^4 kubernetes.io/csi/ebs.csi.aws.com^vol-040ce773b9a5d6014 kubernetes.io/csi/ebs.csi.aws.com^vol-0d656dff677af001a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d656dff677af001a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-7358^4,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6480^8b488b43-ca9d-11ed-948f-b6de23070918,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-040ce773b9a5d6014,DevicePath:,},},Config:nil,},} Mar 24 23:42:57.582: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:42:57.710: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:42:57.812: INFO: test-rolling-update-with-lb-69fcddc54f-42b69 started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:42:57.812: INFO: up-down-3-8nkvj started at 2023-03-24 23:42:51 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container up-down-3 ready: true, restart count 0 Mar 24 23:42:57.812: INFO: hostexec-i-06956c12f825d08d9-8xdhk started at 2023-03-24 23:42:55 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:42:57.812: INFO: inline-volume-sbf9c started at 2023-03-24 23:42:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:42:57.812: INFO: up-down-2-wvmdl started at 2023-03-24 23:42:09 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container up-down-2 ready: true, restart count 0 Mar 24 23:42:57.812: INFO: inline-volume-tester-4whmd started at 2023-03-24 23:42:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:42:57.812: INFO: verify-service-up-host-exec-pod started at 2023-03-24 23:42:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:42:57.812: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:42:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:42:57.812: INFO: liveness-a407d9b9-63aa-4982-be0e-8eb1139ed722 started at 2023-03-24 23:40:58 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:42:57.812: INFO: pod-ready started at 2023-03-24 23:42:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container pod-readiness-gate ready: true, restart count 0 Mar 24 23:42:57.812: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:42:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:42:57.812: INFO: startup-8ffb0034-90d6-4e51-9e22-954487d2fbf1 started at 2023-03-24 23:41:58 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container busybox ready: true, restart count 0 Mar 24 23:42:57.812: INFO: ss2-0 started at 2023-03-24 23:42:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container webserver ready: true, restart count 0 Mar 24 23:42:57.812: INFO: inline-volume-tester-mfnsq started at 2023-03-24 23:42:17 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:42:57.812: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:42:27 +0000 UTC (0+7 container statuses recorded) Mar 24 23:42:57.812: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:57.812: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:57.812: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:57.812: INFO: csi-mockplugin-0 started at 2023-03-24 23:42:26 +0000 UTC (0+4 container statuses recorded) Mar 24 23:42:57.812: INFO: Container busybox ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container mock ready: true, restart count 0 Mar 24 23:42:57.812: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:42:57.812: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:42:57.812: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:42:57.812: INFO: pvc-volume-tester-dn25z started at 2023-03-24 23:42:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:42:57.812: INFO: implicit-nonroot-uid started at 2023-03-24 23:42:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:57.812: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Mar 24 23:42:58.394: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:42:58.394: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:42:58.463: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 29472 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-2377":"i-0713c0b421302f422","ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:42:26 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:42:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:26 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:26 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:42:26 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:42:26 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-2377^69465480-ca9d-11ed-9f74-5ab1c2984cc5],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-2377^69465480-ca9d-11ed-9f74-5ab1c2984cc5,DevicePath:,},},Config:nil,},} Mar 24 23:42:58.464: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:42:58.537: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:42:58.608: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container coredns ready: true, restart count 0 Mar 24 23:42:58.609: INFO: pfpod started at 2023-03-24 23:40:41 +0000 UTC (0+2 container statuses recorded) Mar 24 23:42:58.609: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:42:58.609: INFO: Container readiness ready: false, restart count 0 Mar 24 23:42:58.609: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:41:31 +0000 UTC (0+7 container statuses recorded) Mar 24 23:42:58.609: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:58.609: INFO: ss2-2 started at 2023-03-24 23:42:56 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container webserver ready: true, restart count 0 Mar 24 23:42:58.609: INFO: pod-subpath-test-preprovisionedpv-8ltp started at 2023-03-24 23:42:44 +0000 UTC (1+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Init container init-volume-preprovisionedpv-8ltp ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container test-container-subpath-preprovisionedpv-8ltp ready: true, restart count 0 Mar 24 23:42:58.609: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:42:58.609: INFO: up-down-2-d6sb5 started at 2023-03-24 23:42:09 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container up-down-2 ready: true, restart count 0 Mar 24 23:42:58.609: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:42:58.609: INFO: inline-volume-tester-mwqkj started at 2023-03-24 23:41:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:42:58.609: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:58.609: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:58.609: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:58.609: INFO: hostexec-i-0713c0b421302f422-bk5fr started at 2023-03-24 23:42:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:42:58.609: INFO: up-down-3-xwhzs started at 2023-03-24 23:42:51 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:58.609: INFO: Container up-down-3 ready: true, restart count 0 Mar 24 23:42:58.906: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:42:58.906: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:42:58.979: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 22479 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:40:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:42:58.979: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:42:59.065: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:42:59.151: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:42:59.151: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:42:59.151: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:42:59.151: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:42:59.151: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:42:59.151: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:42:59.151: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:42:59.151: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:42:59.151: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:42:59.151: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:42:59.151: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:59.151: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:42:59.151: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:59.151: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:42:59.151: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.151: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:42:59.151: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:42:59.151: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:42:59.151: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:59.497: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:42:59.497: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:42:59.565: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 26929 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-expansion-3855":"i-0cdea7500a633a81f","csi-mock-csi-mock-volumes-workload-6274":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:41:55 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:41:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:41:58 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:41:58 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:41:58 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:41:58 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-6274^751ae5b5-ca9d-11ed-a34b-96f9604da5d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-6274^751ae5b5-ca9d-11ed-a34b-96f9604da5d9,DevicePath:,},},Config:nil,},} Mar 24 23:42:59.565: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:42:59.642: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:42:59.712: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:42:59.712: INFO: pvc-volume-tester-kg7ms started at 2023-03-24 23:41:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:42:59.712: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:42:59.712: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container coredns ready: true, restart count 0 Mar 24 23:42:59.712: INFO: csi-mockplugin-0 started at 2023-03-24 23:38:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:59.712: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container mock ready: true, restart count 0 Mar 24 23:42:59.712: INFO: pvc-volume-tester-vgkcq started at 2023-03-24 23:38:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:42:59.712: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:42:59.712: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:42:59.712: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:41:48 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:42:59.712: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:59.712: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:42:59.712: INFO: hostexec-i-0cdea7500a633a81f-7lhmf started at 2023-03-24 23:42:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container agnhost-container ready: false, restart count 0 Mar 24 23:42:59.712: INFO: csi-mockplugin-0 started at 2023-03-24 23:41:48 +0000 UTC (0+3 container statuses recorded) Mar 24 23:42:59.712: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:42:59.712: INFO: Container mock ready: true, restart count 0 Mar 24 23:42:59.712: INFO: test-rolling-update-with-lb-69fcddc54f-2hq6t started at 2023-03-24 23:41:36 +0000 UTC (0+1 container statuses recorded) Mar 24 23:42:59.712: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:43:00.030: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:43:00.03 (3.367s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:43:00.03 (3.367s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:43:00.03 STEP: Destroying namespace "provisioning-6035" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:43:00.03 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:43:00.106 (76ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:43:00.106 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:43:00.106 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sdirectory\sspecified\sin\sthe\svolumeMount$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-8xnp for test-container-subpath-dynamicpv-8xnp: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8425/pods/pod-subpath-test-dynamicpv-8xnp/log?container=test-container-subpath-dynamicpv-8xnp&previous=false": stream error: stream ID 2705; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:47:39.073from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:47:23.506 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:47:23.506 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:47:23.506 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:47:23.506 Mar 24 23:47:23.506: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:47:23.507 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:47:23.709 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:47:23.841 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:47:23.972 (467ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:47:23.972 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:47:23.972 (0s) > Enter [It] should support readOnly directory specified in the volumeMount - test/e2e/storage/testsuites/subpath.go:367 @ 03/24/23 23:47:23.972 Mar 24 23:47:23.973: INFO: Creating resource for dynamic PV Mar 24 23:47:23.973: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-8425-e2e-scfhvxn - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:47:23.973 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:47:24.043 Mar 24 23:47:24.043: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-8xnp - test/e2e/storage/testsuites/subpath.go:691 @ 03/24/23 23:47:24.178 STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:47:24.178 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:47:38.798 Mar 24 23:47:38.865: INFO: Trying to get logs from node i-0713c0b421302f422 pod pod-subpath-test-dynamicpv-8xnp container test-container-subpath-dynamicpv-8xnp: <nil> Mar 24 23:47:38.931: INFO: Failed to get logs from node "i-0713c0b421302f422" pod "pod-subpath-test-dynamicpv-8xnp" container "test-container-subpath-dynamicpv-8xnp". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8425/pods/pod-subpath-test-dynamicpv-8xnp/log?container=test-container-subpath-dynamicpv-8xnp&previous=false": stream error: stream ID 2705; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:47:38.931 Mar 24 23:47:39.073: INFO: Unexpected error: <*fmt.wrapError | 0xc002beaa40>: failed to get logs from pod-subpath-test-dynamicpv-8xnp for test-container-subpath-dynamicpv-8xnp: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8425/pods/pod-subpath-test-dynamicpv-8xnp/log?container=test-container-subpath-dynamicpv-8xnp&previous=false": stream error: stream ID 2705; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-8xnp for test-container-subpath-dynamicpv-8xnp: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8425/pods/pod-subpath-test-dynamicpv-8xnp/log?container=test-container-subpath-dynamicpv-8xnp&previous=false\": stream error: stream ID 2705; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc0062b3e60>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8425/pods/pod-subpath-test-dynamicpv-8xnp/log?container=test-container-subpath-dynamicpv-8xnp&previous=false", Err: <http2.StreamError>{ StreamID: 2705, Code: 2, Cause: <*errors.errorString | 0xc0001e76b0>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-8xnp for test-container-subpath-dynamicpv-8xnp: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-8425/pods/pod-subpath-test-dynamicpv-8xnp/log?container=test-container-subpath-dynamicpv-8xnp&previous=false": stream error: stream ID 2705; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:47:39.073 < Exit [It] should support readOnly directory specified in the volumeMount - test/e2e/storage/testsuites/subpath.go:367 @ 03/24/23 23:47:39.073 (15.101s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:47:39.073 Mar 24 23:47:39.073: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:47:39.142 (68ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:369 @ 03/24/23 23:47:39.142 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:47:39.142 Mar 24 23:47:39.142: INFO: Deleting pod "pod-subpath-test-dynamicpv-8xnp" in namespace "provisioning-8425" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:47:39.272 Mar 24 23:47:39.406: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comcfbbx" Mar 24 23:47:39.476: INFO: Waiting up to 5m0s for PersistentVolume pvc-e963dea2-d61e-4dcb-9285-dae23a016175 to get deleted Mar 24 23:47:39.542: INFO: PersistentVolume pvc-e963dea2-d61e-4dcb-9285-dae23a016175 found and phase=Released (66.670738ms) Mar 24 23:47:44.610: INFO: PersistentVolume pvc-e963dea2-d61e-4dcb-9285-dae23a016175 found and phase=Released (5.134555452s) Mar 24 23:47:49.679: INFO: PersistentVolume pvc-e963dea2-d61e-4dcb-9285-dae23a016175 found and phase=Released (10.203124188s) Mar 24 23:47:54.747: INFO: PersistentVolume pvc-e963dea2-d61e-4dcb-9285-dae23a016175 was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:47:54.747 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:369 @ 03/24/23 23:47:54.814 (15.673s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:47:54.814 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:47:54.815 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:47:54.815 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:47:54.815 STEP: Collecting events from namespace "provisioning-8425". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:47:54.815 STEP: Found 15 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:47:54.881 Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:24 +0000 UTC - event for ebs.csi.aws.comcfbbx: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:24 +0000 UTC - event for ebs.csi.aws.comcfbbx: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:24 +0000 UTC - event for ebs.csi.aws.comcfbbx: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-8425/ebs.csi.aws.comcfbbx" Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:27 +0000 UTC - event for ebs.csi.aws.comcfbbx: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-e963dea2-d61e-4dcb-9285-dae23a016175 Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:28 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {default-scheduler } Scheduled: Successfully assigned provisioning-8425/pod-subpath-test-dynamicpv-8xnp to i-0713c0b421302f422 Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:30 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-e963dea2-d61e-4dcb-9285-dae23a016175" Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:34 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:34 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Created: Created container init-volume-dynamicpv-8xnp Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:34 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Started: Started container init-volume-dynamicpv-8xnp Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:35 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:35 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Created: Created container test-init-volume-dynamicpv-8xnp Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:35 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Started: Started container test-init-volume-dynamicpv-8xnp Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:36 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:36 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Created: Created container test-container-subpath-dynamicpv-8xnp Mar 24 23:47:54.881: INFO: At 2023-03-24 23:47:36 +0000 UTC - event for pod-subpath-test-dynamicpv-8xnp: {kubelet i-0713c0b421302f422} Started: Started container test-container-subpath-dynamicpv-8xnp Mar 24 23:47:54.947: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:47:54.947: INFO: Mar 24 23:47:55.016: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:47:55.082: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 43007 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-7616":"i-005794773d640afd9","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:47:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:51 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:51 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:51 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:47:51 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:47:55.083: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:47:55.157: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:47:55.238: INFO: busybox-da90c053-5779-4b82-925b-3e9d2cb32681 started at 2023-03-24 23:47:03 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container busybox ready: true, restart count 0 Mar 24 23:47:55.238: INFO: inline-volume-tester-blzbc started at 2023-03-24 23:47:42 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:47:55.238: INFO: affinity-nodeport-transition-2xhxj started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 24 23:47:55.238: INFO: pod-with-poststart-http-hook started at 2023-03-24 23:47:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container pod-with-poststart-http-hook ready: true, restart count 0 Mar 24 23:47:55.238: INFO: simpletest.rc-vsqhp started at 2023-03-24 23:47:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container nginx ready: true, restart count 0 Mar 24 23:47:55.238: INFO: inline-volume-tester2-f76w7 started at 2023-03-24 23:47:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:47:55.238: INFO: pod-handle-http-request started at 2023-03-24 23:47:13 +0000 UTC (0+2 container statuses recorded) Mar 24 23:47:55.238: INFO: Container container-handle-http-request ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container container-handle-https-request ready: true, restart count 0 Mar 24 23:47:55.238: INFO: netserver-0 started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container webserver ready: false, restart count 0 Mar 24 23:47:55.238: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:47:55.238: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:47:55.238: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:47:55.238: INFO: annotationupdated6d1ee1a-7c07-45ab-8836-dd4eb54be843 started at 2023-03-24 23:47:28 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container client-container ready: true, restart count 0 Mar 24 23:47:55.238: INFO: inline-volume-wkknp started at 2023-03-24 23:46:09 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:47:55.238: INFO: pod-projected-configmaps-cd5dc486-c3e2-4aed-b2e5-a109d691d2a4 started at 2023-03-24 23:47:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:47:55.238: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:55.238: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:55.238: INFO: annotationupdate1566e306-b227-4b68-80c2-f0b730a497a8 started at 2023-03-24 23:47:41 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.238: INFO: Container client-container ready: true, restart count 0 Mar 24 23:47:55.238: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:42 +0000 UTC (0+7 container statuses recorded) Mar 24 23:47:55.238: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:55.238: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:55.643: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:47:55.643: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:47:55.710: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 42750 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-read-write-once-pod-3668":"i-06956c12f825d08d9","csi-mock-csi-mock-volumes-workload-6997":"i-06956c12f825d08d9","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:47:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:47:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:47:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-05626d827e3c019cb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-05626d827e3c019cb,DevicePath:,},},Config:nil,},} Mar 24 23:47:55.710: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:47:55.785: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:47:55.858: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:12 +0000 UTC (0+7 container statuses recorded) Mar 24 23:47:55.858: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:55.858: INFO: pvc-volume-tester-hfx4d started at 2023-03-24 23:46:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.858: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:47:55.858: INFO: pod-secrets-89f39212-2f92-452d-a143-8ea5b1f722b0 started at 2023-03-24 23:43:46 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:55.858: INFO: Container creates-volume-test ready: false, restart count 0 Mar 24 23:47:55.858: INFO: Container dels-volume-test ready: false, restart count 0 Mar 24 23:47:55.858: INFO: Container upds-volume-test ready: false, restart count 0 Mar 24 23:47:55.858: INFO: csi-mockplugin-0 started at 2023-03-24 23:46:01 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:55.858: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container mock ready: true, restart count 0 Mar 24 23:47:55.858: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:46:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.858: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:47:55.858: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:55.858: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:55.858: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:55.858: INFO: affinity-nodeport-transition-6plm6 started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.858: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 24 23:47:55.858: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.858: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:47:55.859: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:47:55.859: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:47:55.859: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:47:55.859: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:47:55.859: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:47:55.859: INFO: inline-volume-tester-wpdjn started at 2023-03-24 23:47:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.859: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:47:55.859: INFO: netserver-1 started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:55.859: INFO: Container webserver ready: false, restart count 0 Mar 24 23:47:55.859: INFO: dns-test-51a345c0-56d4-4b6b-9a93-7082f5e09fe4 started at 2023-03-24 23:47:35 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:55.859: INFO: Container jessie-querier ready: true, restart count 0 Mar 24 23:47:55.859: INFO: Container querier ready: true, restart count 0 Mar 24 23:47:55.859: INFO: Container webserver ready: true, restart count 0 Mar 24 23:47:56.255: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:47:56.255: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:47:56.325: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 43126 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3735":"i-0713c0b421302f422","ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:47:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:47:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:47:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3735^3c64aba5-ca9e-11ed-934b-76555be6a59a kubernetes.io/csi/csi-hostpath-ephemeral-3735^425f522a-ca9e-11ed-934b-76555be6a59a kubernetes.io/csi/ebs.csi.aws.com^vol-055d4b9f9ed0ba6b0],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3735^3c64aba5-ca9e-11ed-934b-76555be6a59a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-055d4b9f9ed0ba6b0,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3735^425f522a-ca9e-11ed-934b-76555be6a59a,DevicePath:,},},Config:nil,},} Mar 24 23:47:56.325: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:47:56.408: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:47:56.481: INFO: netserver-2 started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container webserver ready: false, restart count 0 Mar 24 23:47:56.481: INFO: inline-volume-tester-2fgw7 started at 2023-03-24 23:47:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:47:56.481: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container coredns ready: true, restart count 0 Mar 24 23:47:56.481: INFO: inline-volume-tester2-bb52t started at 2023-03-24 23:47:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:47:56.481: INFO: hostexec-i-0713c0b421302f422-kdhvs started at 2023-03-24 23:47:48 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:47:56.481: INFO: pod-subpath-test-dynamicpv-jmfk started at 2023-03-24 23:47:46 +0000 UTC (1+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Init container init-volume-dynamicpv-jmfk ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container test-container-subpath-dynamicpv-jmfk ready: true, restart count 0 Mar 24 23:47:56.481: INFO: pfpod started at 2023-03-24 23:47:00 +0000 UTC (0+2 container statuses recorded) Mar 24 23:47:56.481: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:47:56.481: INFO: Container readiness ready: false, restart count 0 Mar 24 23:47:56.481: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:47:26 +0000 UTC (0+7 container statuses recorded) Mar 24 23:47:56.481: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:56.481: INFO: startup-da1780a0-61c5-4110-8f42-ae25c8c37b21 started at 2023-03-24 23:47:44 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container busybox ready: false, restart count 0 Mar 24 23:47:56.481: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:47:56.481: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:56.481: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:47:56.481: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:56.481: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:56.481: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:56.793: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:47:56.793: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:47:56.861: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 36523 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:45:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:45:21 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:47:56.862: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:47:56.935: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:47:57.014: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:47:57.014: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:47:57.014: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:47:57.014: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:47:57.014: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:47:57.014: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:47:57.014: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:47:57.014: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:47:57.014: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:47:57.014: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:47:57.014: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:57.014: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:47:57.014: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:57.014: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:47:57.014: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.014: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:47:57.014: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:47:57.014: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:47:57.014: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:57.364: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:47:57.364: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:47:57.431: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 42964 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:47:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:47:45 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:47:57.431: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:47:57.504: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:47:57.577: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:47:57.577: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:47:57.577: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.577: INFO: Container coredns ready: true, restart count 0 Mar 24 23:47:57.577: INFO: affinity-nodeport-transition-r6bnw started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.577: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 24 23:47:57.577: INFO: pod-configmaps-4faa40ec-e6e6-4f18-a7f9-fbad3cc24f7e started at 2023-03-24 23:46:35 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:57.577: INFO: Container createcm-volume-test ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Container delcm-volume-test ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Container updcm-volume-test ready: true, restart count 0 Mar 24 23:47:57.577: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.577: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:47:57.577: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:47:57.577: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:47:57.577: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:47:57.577: INFO: simpletest.rc-6prhs started at 2023-03-24 23:47:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.577: INFO: Container nginx ready: true, restart count 0 Mar 24 23:47:57.577: INFO: netserver-3 started at 2023-03-24 23:47:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:47:57.577: INFO: Container webserver ready: false, restart count 0 Mar 24 23:47:57.894: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:47:57.894 (3.079s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:47:57.894 (3.079s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:47:57.894 STEP: Destroying namespace "provisioning-8425" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:47:57.894 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:47:57.963 (69ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:47:57.963 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:47:57.963 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sfile\sspecified\sin\sthe\svolumeMount\s\[LinuxOnly\]$'
[FAILED] failed to get logs from pod-subpath-test-dynamicpv-7khh for test-container-subpath-dynamicpv-7khh: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-941/pods/pod-subpath-test-dynamicpv-7khh/log?container=test-container-subpath-dynamicpv-7khh&previous=false": stream error: stream ID 115; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:36:44.575from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:36:31.101 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:36:31.101 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:36:31.101 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:36:31.101 Mar 24 23:36:31.101: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/24/23 23:36:31.103 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:36:31.298 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:36:31.426 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - set up framework | framework.go:191 @ 03/24/23 23:36:31.556 (455ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:36:31.556 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:36:31.557 (0s) > Enter [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/24/23 23:36:31.557 Mar 24 23:36:31.557: INFO: Creating resource for dynamic PV Mar 24 23:36:31.557: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass provisioning-941-e2e-scp2njr - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:36:31.558 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:36:31.629 Mar 24 23:36:31.629: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod pod-subpath-test-dynamicpv-7khh - test/e2e/storage/testsuites/subpath.go:691 @ 03/24/23 23:36:31.764 STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:36:31.765 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:36:44.305 Mar 24 23:36:44.369: INFO: Trying to get logs from node i-005794773d640afd9 pod pod-subpath-test-dynamicpv-7khh container test-container-subpath-dynamicpv-7khh: <nil> Mar 24 23:36:44.434: INFO: Failed to get logs from node "i-005794773d640afd9" pod "pod-subpath-test-dynamicpv-7khh" container "test-container-subpath-dynamicpv-7khh". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-941/pods/pod-subpath-test-dynamicpv-7khh/log?container=test-container-subpath-dynamicpv-7khh&previous=false": stream error: stream ID 115; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:36:44.434 Mar 24 23:36:44.575: INFO: Unexpected error: <*fmt.wrapError | 0xc005fed900>: failed to get logs from pod-subpath-test-dynamicpv-7khh for test-container-subpath-dynamicpv-7khh: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-941/pods/pod-subpath-test-dynamicpv-7khh/log?container=test-container-subpath-dynamicpv-7khh&previous=false": stream error: stream ID 115; INTERNAL_ERROR; received from peer { msg: "failed to get logs from pod-subpath-test-dynamicpv-7khh for test-container-subpath-dynamicpv-7khh: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-941/pods/pod-subpath-test-dynamicpv-7khh/log?container=test-container-subpath-dynamicpv-7khh&previous=false\": stream error: stream ID 115; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc0044c8ea0>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-941/pods/pod-subpath-test-dynamicpv-7khh/log?container=test-container-subpath-dynamicpv-7khh&previous=false", Err: <http2.StreamError>{ StreamID: 115, Code: 2, Cause: <*errors.errorString | 0xc0001b3690>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from pod-subpath-test-dynamicpv-7khh for test-container-subpath-dynamicpv-7khh: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-941/pods/pod-subpath-test-dynamicpv-7khh/log?container=test-container-subpath-dynamicpv-7khh&previous=false": stream error: stream ID 115; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:36:44.575 < Exit [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/24/23 23:36:44.575 (13.019s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:36:44.575 Mar 24 23:36:44.575: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:36:44.64 (65ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:384 @ 03/24/23 23:36:44.64 STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/24/23 23:36:44.64 Mar 24 23:36:44.640: INFO: Deleting pod "pod-subpath-test-dynamicpv-7khh" in namespace "provisioning-941" STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:36:44.765 Mar 24 23:36:44.898: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comk4lks" Mar 24 23:36:44.968: INFO: Waiting up to 5m0s for PersistentVolume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb to get deleted Mar 24 23:36:45.038: INFO: PersistentVolume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb found and phase=Released (70.126017ms) Mar 24 23:36:50.108: INFO: PersistentVolume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb found and phase=Released (5.140423823s) Mar 24 23:36:55.173: INFO: PersistentVolume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb found and phase=Released (10.205336808s) Mar 24 23:37:00.293: INFO: PersistentVolume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb found and phase=Released (15.325310911s) Mar 24 23:37:05.360: INFO: PersistentVolume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:37:05.36 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:384 @ 03/24/23 23:37:05.428 (20.788s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:37:05.428 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:37:05.428 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:37:05.428 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:37:05.428 STEP: Collecting events from namespace "provisioning-941". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:37:05.428 STEP: Found 15 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:37:05.497 Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:31 +0000 UTC - event for ebs.csi.aws.comk4lks: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:31 +0000 UTC - event for ebs.csi.aws.comk4lks: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:31 +0000 UTC - event for ebs.csi.aws.comk4lks: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "provisioning-941/ebs.csi.aws.comk4lks" Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:35 +0000 UTC - event for ebs.csi.aws.comk4lks: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-5860dbeb-79da-490c-bb35-8937b5f504cb Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:35 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {default-scheduler } Scheduled: Successfully assigned provisioning-941/pod-subpath-test-dynamicpv-7khh to i-005794773d640afd9 Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:38 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-5860dbeb-79da-490c-bb35-8937b5f504cb" Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:39 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:39 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Created: Created container init-volume-dynamicpv-7khh Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:39 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Started: Started container init-volume-dynamicpv-7khh Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:39 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:39 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Created: Created container test-init-volume-dynamicpv-7khh Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:39 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Started: Started container test-init-volume-dynamicpv-7khh Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:40 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:40 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Created: Created container test-container-subpath-dynamicpv-7khh Mar 24 23:37:05.497: INFO: At 2023-03-24 23:36:40 +0000 UTC - event for pod-subpath-test-dynamicpv-7khh: {kubelet i-005794773d640afd9} Started: Started container test-container-subpath-dynamicpv-7khh Mar 24 23:37:05.575: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:37:05.575: INFO: Mar 24 23:37:05.644: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:37:05.713: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 8658 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-9249":"i-005794773d640afd9","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:36:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:36:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:47 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:47 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:47 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:36:47 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-9249^afa44bf5-ca9c-11ed-8a7f-d2d9b50df69b kubernetes.io/csi/csi-hostpath-ephemeral-9249^b50b23a5-ca9c-11ed-8a7f-d2d9b50df69b kubernetes.io/csi/ebs.csi.aws.com^vol-007ae6e497a3607e1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-007ae6e497a3607e1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-9249^afa44bf5-ca9c-11ed-8a7f-d2d9b50df69b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-9249^b50b23a5-ca9c-11ed-8a7f-d2d9b50df69b,DevicePath:,},},Config:nil,},} Mar 24 23:37:05.713: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:37:05.800: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:37:05.953: INFO: pod1 started at 2023-03-24 23:37:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container agnhost ready: false, restart count 0 Mar 24 23:37:05.953: INFO: inline-volume-tester2-pm4gc started at 2023-03-24 23:36:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:37:05.953: INFO: test-orphan-deployment-67bd4bf6dc-lfcvr started at 2023-03-24 23:36:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container httpd ready: true, restart count 0 Mar 24 23:37:05.953: INFO: sysctl-24af73e0-bbed-418d-9984-c6c37f5f9372 started at 2023-03-24 23:36:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container test-container ready: false, restart count 0 Mar 24 23:37:05.953: INFO: ss-0 started at 2023-03-24 23:36:13 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container webserver ready: true, restart count 0 Mar 24 23:37:05.953: INFO: execpodlnhdv started at 2023-03-24 23:36:57 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:37:05.953: INFO: startup-8fdbdc1a-89d6-4268-84b7-a6654ac09481 started at 2023-03-24 23:36:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container busybox ready: false, restart count 0 Mar 24 23:37:05.953: INFO: inline-volume-tester-hlxt2 started at 2023-03-24 23:36:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:37:05.953: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:36:13 +0000 UTC (0+7 container statuses recorded) Mar 24 23:37:05.953: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:05.953: INFO: rs-mkpzh started at 2023-03-24 23:36:58 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container donothing ready: false, restart count 0 Mar 24 23:37:05.953: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:37:05.953: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:37:05.953: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:05.953: INFO: netserver-0 started at 2023-03-24 23:37:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container webserver ready: false, restart count 0 Mar 24 23:37:05.953: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:37:05.953: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:37:05.953: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:37:05.953: INFO: busybox-c09a66e2-3e82-43de-8fb7-6403b63c1ccc started at 2023-03-24 23:36:37 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:05.953: INFO: Container busybox ready: true, restart count 0 Mar 24 23:37:06.313: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:37:06.313: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:37:06.379: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 9996 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-attach-1873":"i-06956c12f825d08d9","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:36:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:37:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:37:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:37:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:37:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:37:00 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-attach-1873^c2c3b26d-ca9c-11ed-a0f1-fe4e1610c284 kubernetes.io/csi/ebs.csi.aws.com^vol-05f55a1f7ab3a0230],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-05f55a1f7ab3a0230,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-attach-1873^c2c3b26d-ca9c-11ed-a0f1-fe4e1610c284,DevicePath:,},},Config:nil,},} Mar 24 23:37:06.380: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:37:06.456: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:37:06.533: INFO: hostexec-i-06956c12f825d08d9-9kfxd started at 2023-03-24 23:36:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:37:06.533: INFO: liveness-b6c3a35c-3df1-40cc-adc9-46b201d3570a started at 2023-03-24 23:36:11 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container agnhost-container ready: true, restart count 2 Mar 24 23:37:06.533: INFO: hostexec-i-06956c12f825d08d9-h2zbf started at 2023-03-24 23:36:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:37:06.533: INFO: ss-1 started at 2023-03-24 23:36:33 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container webserver ready: true, restart count 0 Mar 24 23:37:06.533: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:37:06.533: INFO: bin-falseb731d9a2-cfa0-4410-ba23-b0f1f8e725b0 started at 2023-03-24 23:37:02 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container bin-falseb731d9a2-cfa0-4410-ba23-b0f1f8e725b0 ready: false, restart count 0 Mar 24 23:37:06.533: INFO: pod-b41de85f-3ed2-42a9-99c4-e2c6cfc5348c started at 2023-03-24 23:36:31 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container write-pod ready: true, restart count 0 Mar 24 23:37:06.533: INFO: csi-mockplugin-0 started at 2023-03-24 23:36:45 +0000 UTC (0+3 container statuses recorded) Mar 24 23:37:06.533: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Container mock ready: true, restart count 0 Mar 24 23:37:06.533: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:37:06.533: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:06.533: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:06.533: INFO: pfpod started at 2023-03-24 23:36:07 +0000 UTC (0+2 container statuses recorded) Mar 24 23:37:06.533: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:37:06.533: INFO: Container readiness ready: false, restart count 0 Mar 24 23:37:06.533: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:36:45 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:37:06.533: INFO: netserver-1 started at 2023-03-24 23:37:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container webserver ready: false, restart count 0 Mar 24 23:37:06.533: INFO: var-expansion-97e2177c-f0c7-4b23-8b41-5d49083bc91b started at 2023-03-24 23:37:03 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container dapi-container ready: false, restart count 0 Mar 24 23:37:06.533: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:37:06.533: INFO: pod-2ebd70c2-86c9-4132-b3e4-33b580c4a784 started at 2023-03-24 23:37:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:06.533: INFO: Container write-pod ready: true, restart count 0 Mar 24 23:37:06.815: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:37:06.815: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:37:06.880: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 10101 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-9164":"i-0713c0b421302f422","ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:36:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:37:02 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:58 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:58 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:58 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:36:58 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0edf75b00eb0b02bc],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0edf75b00eb0b02bc,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-07cd67779395404e5,DevicePath:,},},Config:nil,},} Mar 24 23:37:06.880: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:37:06.968: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:37:07.042: INFO: pod-terminate-status-1-0 started at 2023-03-24 23:37:05 +0000 UTC (1+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Init container fail ready: false, restart count 0 Mar 24 23:37:07.042: INFO: Container blocked ready: false, restart count 0 Mar 24 23:37:07.042: INFO: pod-terminate-status-2-0 started at 2023-03-24 23:37:05 +0000 UTC (1+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Init container fail ready: false, restart count 0 Mar 24 23:37:07.042: INFO: Container blocked ready: false, restart count 0 Mar 24 23:37:07.042: INFO: nodeport-test-d7gdg started at 2023-03-24 23:36:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container nodeport-test ready: true, restart count 0 Mar 24 23:37:07.042: INFO: ss-2 started at 2023-03-24 23:36:55 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container webserver ready: true, restart count 0 Mar 24 23:37:07.042: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:37:07.042: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container coredns ready: true, restart count 0 Mar 24 23:37:07.042: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:37:07.042: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:37:07.042: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:07.042: INFO: netserver-2 started at 2023-03-24 23:37:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container webserver ready: false, restart count 0 Mar 24 23:37:07.042: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:36:30 +0000 UTC (0+7 container statuses recorded) Mar 24 23:37:07.042: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:07.042: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:07.042: INFO: all-succeed-vhswk started at 2023-03-24 23:37:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container c ready: false, restart count 0 Mar 24 23:37:07.042: INFO: pod-terminate-status-0-0 started at 2023-03-24 23:37:05 +0000 UTC (1+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Init container fail ready: false, restart count 0 Mar 24 23:37:07.042: INFO: Container blocked ready: false, restart count 0 Mar 24 23:37:07.042: INFO: external-injector started at 2023-03-24 23:36:59 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.042: INFO: Container external-injector ready: false, restart count 0 Mar 24 23:37:07.516: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:37:07.516: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:37:07.581: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 5810 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:35:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:37:07.581: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:37:07.667: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:37:07.739: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:37:07.739: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:37:07.739: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:37:07.739: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:37:07.739: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:07.739: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:37:07.739: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:37:07.739: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:37:07.739: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:37:07.739: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:37:07.739: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:37:07.739: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:07.739: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:37:07.739: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:37:07.739: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:37:07.739: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:37:07.739: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:37:07.739: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:07.739: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:37:08.057: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:37:08.057: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:37:08.123: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 8339 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-7206":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:36:41 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:36:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:41 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:41 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:36:41 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:36:41 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-7206^b64c9e41-ca9c-11ed-8d59-ca36e4a58c3b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-7206^b64c9e41-ca9c-11ed-8d59-ca36e4a58c3b,DevicePath:,},},Config:nil,},} Mar 24 23:37:08.123: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:37:08.245: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:37:08.338: INFO: nodeport-test-s5xfv started at 2023-03-24 23:36:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container nodeport-test ready: true, restart count 0 Mar 24 23:37:08.338: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:37:08.338: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:08.338: INFO: hostexec-i-0cdea7500a633a81f-2szbd started at 2023-03-24 23:37:05 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:37:08.338: INFO: inline-volume-tester-hpkxm started at 2023-03-24 23:36:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:37:08.338: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:36:25 +0000 UTC (0+7 container statuses recorded) Mar 24 23:37:08.338: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:37:08.338: INFO: netserver-3 started at 2023-03-24 23:37:01 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container webserver ready: false, restart count 0 Mar 24 23:37:08.338: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:37:08.338: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:37:08.338: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:37:08.338: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container coredns ready: true, restart count 0 Mar 24 23:37:08.338: INFO: pfpod started at 2023-03-24 23:36:08 +0000 UTC (0+2 container statuses recorded) Mar 24 23:37:08.338: INFO: Container portforwardtester ready: false, restart count 0 Mar 24 23:37:08.338: INFO: Container readiness ready: false, restart count 0 Mar 24 23:37:08.338: INFO: agnhost-primary-x4jjq started at 2023-03-24 23:36:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:37:08.338: INFO: Container agnhost-primary ready: false, restart count 0 Mar 24 23:37:08.645: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:37:08.645 (3.217s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/24/23 23:37:08.645 (3.217s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:37:08.645 STEP: Destroying namespace "provisioning-941" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:37:08.645 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/24/23 23:37:08.711 (67ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:37:08.711 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:37:08.712 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\svolumes\sshould\sallow\sexec\sof\sfiles\son\sthe\svolume$'
[FAILED] failed to get logs from exec-volume-test-dynamicpv-2cdq for exec-container-dynamicpv-2cdq: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-9856/pods/exec-volume-test-dynamicpv-2cdq/log?container=exec-container-dynamicpv-2cdq&previous=false": stream error: stream ID 2721; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:53:04.631from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:52:46.967 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:52:46.967 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes - set up framework | framework.go:191 @ 03/24/23 23:52:46.967 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:52:46.967 Mar 24 23:52:46.967: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename volume - test/e2e/framework/framework.go:250 @ 03/24/23 23:52:46.968 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:52:47.171 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:52:47.313 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes - set up framework | framework.go:191 @ 03/24/23 23:52:47.45 (483ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:52:47.45 < Exit [BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:52:47.45 (0s) > Enter [It] should allow exec of files on the volume - test/e2e/storage/testsuites/volumes.go:194 @ 03/24/23 23:52:47.45 Mar 24 23:52:47.450: INFO: Creating resource for dynamic PV Mar 24 23:52:47.450: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass volume-9856-e2e-scqm5q2 - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:52:47.451 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:52:47.532 Mar 24 23:52:47.532: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod exec-volume-test-dynamicpv-2cdq - test/e2e/storage/testsuites/volumes.go:253 @ 03/24/23 23:52:47.673 STEP: Creating a pod to test exec-volume-test - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:52:47.673 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:53:04.349 Mar 24 23:53:04.413: INFO: Trying to get logs from node i-06956c12f825d08d9 pod exec-volume-test-dynamicpv-2cdq container exec-container-dynamicpv-2cdq: <nil> Mar 24 23:53:04.478: INFO: Failed to get logs from node "i-06956c12f825d08d9" pod "exec-volume-test-dynamicpv-2cdq" container "exec-container-dynamicpv-2cdq". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-9856/pods/exec-volume-test-dynamicpv-2cdq/log?container=exec-container-dynamicpv-2cdq&previous=false": stream error: stream ID 2721; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:53:04.478 Mar 24 23:53:04.631: INFO: Unexpected error: <*fmt.wrapError | 0xc000ee07e0>: failed to get logs from exec-volume-test-dynamicpv-2cdq for exec-container-dynamicpv-2cdq: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-9856/pods/exec-volume-test-dynamicpv-2cdq/log?container=exec-container-dynamicpv-2cdq&previous=false": stream error: stream ID 2721; INTERNAL_ERROR; received from peer { msg: "failed to get logs from exec-volume-test-dynamicpv-2cdq for exec-container-dynamicpv-2cdq: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-9856/pods/exec-volume-test-dynamicpv-2cdq/log?container=exec-container-dynamicpv-2cdq&previous=false\": stream error: stream ID 2721; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc0044a2780>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-9856/pods/exec-volume-test-dynamicpv-2cdq/log?container=exec-container-dynamicpv-2cdq&previous=false", Err: <http2.StreamError>{ StreamID: 2721, Code: 2, Cause: <*errors.errorString | 0xc0001e9680>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from exec-volume-test-dynamicpv-2cdq for exec-container-dynamicpv-2cdq: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-9856/pods/exec-volume-test-dynamicpv-2cdq/log?container=exec-container-dynamicpv-2cdq&previous=false": stream error: stream ID 2721; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:53:04.631 < Exit [It] should allow exec of files on the volume - test/e2e/storage/testsuites/volumes.go:194 @ 03/24/23 23:53:04.631 (17.182s) > Enter [AfterEach] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:53:04.631 Mar 24 23:53:04.631: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:53:04.706 (75ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/storage/testsuites/volumes.go:197 @ 03/24/23 23:53:04.706 STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:53:04.706 Mar 24 23:53:04.836: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comdkmgt" Mar 24 23:53:04.903: INFO: Waiting up to 5m0s for PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 to get deleted Mar 24 23:53:04.968: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (64.741538ms) Mar 24 23:53:10.034: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (5.130851988s) Mar 24 23:53:15.108: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (10.204798438s) Mar 24 23:53:20.175: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (15.272057649s) Mar 24 23:53:25.241: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (20.338329709s) Mar 24 23:53:30.309: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (25.405646728s) Mar 24 23:53:35.376: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 found and phase=Released (30.47256615s) Mar 24 23:53:40.442: INFO: PersistentVolume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:53:40.442 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/storage/testsuites/volumes.go:197 @ 03/24/23 23:53:40.51 (35.804s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:53:40.51 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:53:40.51 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - dump namespaces | framework.go:209 @ 03/24/23 23:53:40.51 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:53:40.51 STEP: Collecting events from namespace "volume-9856". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:53:40.51 STEP: Found 9 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:53:40.578 Mar 24 23:53:40.578: INFO: At 2023-03-24 23:52:47 +0000 UTC - event for ebs.csi.aws.comdkmgt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:53:40.578: INFO: At 2023-03-24 23:52:47 +0000 UTC - event for ebs.csi.aws.comdkmgt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:53:40.578: INFO: At 2023-03-24 23:52:47 +0000 UTC - event for ebs.csi.aws.comdkmgt: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "volume-9856/ebs.csi.aws.comdkmgt" Mar 24 23:53:40.578: INFO: At 2023-03-24 23:52:51 +0000 UTC - event for ebs.csi.aws.comdkmgt: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-957faa90-33ad-4a5a-81d2-4da776808c51 Mar 24 23:53:40.578: INFO: At 2023-03-24 23:52:51 +0000 UTC - event for exec-volume-test-dynamicpv-2cdq: {default-scheduler } Scheduled: Successfully assigned volume-9856/exec-volume-test-dynamicpv-2cdq to i-06956c12f825d08d9 Mar 24 23:53:40.578: INFO: At 2023-03-24 23:52:59 +0000 UTC - event for exec-volume-test-dynamicpv-2cdq: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-957faa90-33ad-4a5a-81d2-4da776808c51" Mar 24 23:53:40.578: INFO: At 2023-03-24 23:53:00 +0000 UTC - event for exec-volume-test-dynamicpv-2cdq: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/nginx:1.14-4" already present on machine Mar 24 23:53:40.578: INFO: At 2023-03-24 23:53:00 +0000 UTC - event for exec-volume-test-dynamicpv-2cdq: {kubelet i-06956c12f825d08d9} Created: Created container exec-container-dynamicpv-2cdq Mar 24 23:53:40.578: INFO: At 2023-03-24 23:53:00 +0000 UTC - event for exec-volume-test-dynamicpv-2cdq: {kubelet i-06956c12f825d08d9} Started: Started container exec-container-dynamicpv-2cdq Mar 24 23:53:40.644: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:53:40.644: INFO: Mar 24 23:53:40.717: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:53:40.849: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 60865 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:53:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-080957e38bee7cecb],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:53:40.850: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:53:40.931: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:53:41.010: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:41.010: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:41.010: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:41.010: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:41.010: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:53:41.010: INFO: httpd started at 2023-03-24 23:53:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container httpd ready: true, restart count 0 Mar 24 23:53:41.010: INFO: pod-subpath-test-inlinevolume-5w98 started at 2023-03-24 23:53:36 +0000 UTC (0+2 container statuses recorded) Mar 24 23:53:41.010: INFO: Container test-container-subpath-inlinevolume-5w98 ready: false, restart count 0 Mar 24 23:53:41.010: INFO: Container test-container-volume-inlinevolume-5w98 ready: true, restart count 0 Mar 24 23:53:41.010: INFO: indexed-job-1-nktzj started at 2023-03-24 23:53:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container c ready: false, restart count 0 Mar 24 23:53:41.010: INFO: pause-pod-1 started at 2023-03-24 23:53:18 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:41.010: INFO: indexed-job-0-t658r started at 2023-03-24 23:53:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container c ready: false, restart count 0 Mar 24 23:53:41.010: INFO: security-context-4b57b7f6-bbe4-476e-84f9-608664ae965a started at 2023-03-24 23:53:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container test-container ready: false, restart count 0 Mar 24 23:53:41.010: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:41.010: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:41.010: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:41.010: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:41.010: INFO: ss-2 started at 2023-03-24 23:52:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container webserver ready: false, restart count 0 Mar 24 23:53:41.010: INFO: service-proxy-toggled-25qww started at 2023-03-24 23:53:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container service-proxy-toggled ready: true, restart count 0 Mar 24 23:53:41.010: INFO: ephemeral-containers-target-pod started at 2023-03-24 23:53:18 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container test-container-1 ready: true, restart count 0 Mar 24 23:53:41.010: INFO: pod-subpath-test-projected-88jj started at 2023-03-24 23:53:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container test-container-subpath-projected-88jj ready: true, restart count 0 Mar 24 23:53:41.010: INFO: service-proxy-disabled-6nspv started at 2023-03-24 23:53:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container service-proxy-disabled ready: true, restart count 0 Mar 24 23:53:41.010: INFO: webserver-pod started at 2023-03-24 23:51:23 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container agnhost ready: false, restart count 0 Mar 24 23:53:41.010: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:41.010: INFO: webserver-pod started at 2023-03-24 23:53:16 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.010: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:41.338: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:53:41.338: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:53:41.404: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 61679 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:53:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:53:25 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0240efc31331fc483],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0240efc31331fc483,DevicePath:,},},Config:nil,},} Mar 24 23:53:41.405: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:53:41.479: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:53:41.552: INFO: downwardapi-volume-ec75b8a5-f4de-408f-bcc7-d15056cfb768 started at 2023-03-24 23:53:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container client-container ready: false, restart count 0 Mar 24 23:53:41.552: INFO: indexed-job-3-zhlq8 started at 2023-03-24 23:53:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container c ready: false, restart count 0 Mar 24 23:53:41.552: INFO: service-proxy-toggled-sh662 started at 2023-03-24 23:53:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container service-proxy-toggled ready: true, restart count 0 Mar 24 23:53:41.552: INFO: pod-configmaps-5e207db0-ceab-4dc0-81c4-130e29bd58ba started at 2023-03-24 23:49:48 +0000 UTC (0+2 container statuses recorded) Mar 24 23:53:41.552: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:41.552: INFO: Container configmap-volume-binary-test ready: false, restart count 0 Mar 24 23:53:41.552: INFO: test-pod-1 started at 2023-03-24 23:53:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container token-test ready: true, restart count 0 Mar 24 23:53:41.552: INFO: ss-1 started at 2023-03-24 23:52:25 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container webserver ready: true, restart count 0 Mar 24 23:53:41.552: INFO: pod-subpath-test-configmap-sldt started at 2023-03-24 23:53:19 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container test-container-subpath-configmap-sldt ready: false, restart count 0 Mar 24 23:53:41.552: INFO: indexed-job-2-4mbcq started at 2023-03-24 23:53:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container c ready: false, restart count 0 Mar 24 23:53:41.552: INFO: test-pod-2 started at 2023-03-24 23:53:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container token-test ready: true, restart count 0 Mar 24 23:53:41.552: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:41.552: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:41.552: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:41.552: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:41.552: INFO: test-pod-3 started at 2023-03-24 23:53:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container token-test ready: true, restart count 0 Mar 24 23:53:41.552: INFO: service-proxy-disabled-49qrd started at 2023-03-24 23:53:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container service-proxy-disabled ready: true, restart count 0 Mar 24 23:53:41.552: INFO: backofflimit-j9bgg started at 2023-03-24 23:53:33 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container c ready: false, restart count 0 Mar 24 23:53:41.552: INFO: pod-csi-inline-volumes started at 2023-03-24 23:52:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 24 23:53:41.552: INFO: pod-configmaps-267d1d9e-c0fc-45e7-b918-870e2680fc24 started at 2023-03-24 23:50:20 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:41.552: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:41.552: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:41.552: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:41.552: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:41.552: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:41.552: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:41.860: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:53:41.860: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:53:41.926: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 62454 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-4263":"i-0713c0b421302f422","csi-mock-csi-mock-volumes-capacity-6361":"csi-mock-csi-mock-volumes-capacity-6361","csi-mock-csi-mock-volumes-expansion-4869":"i-0713c0b421302f422","ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:53:40 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:53:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:40 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:40 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:40 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:53:40 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-4869^0ea4c7bc-ca9f-11ed-b0ba-723793da46e9 kubernetes.io/csi/ebs.csi.aws.com^vol-00477843dedaa7dfd],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-00477843dedaa7dfd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-4869^0ea4c7bc-ca9f-11ed-b0ba-723793da46e9,DevicePath:,},},Config:nil,},} Mar 24 23:53:41.927: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:53:42.004: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:53:42.084: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:42.084: INFO: pvc-volume-tester-ch6jr started at 2023-03-24 23:53:21 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:53:42.084: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:42.084: INFO: csi-mockplugin-0 started at 2023-03-24 23:53:13 +0000 UTC (0+4 container statuses recorded) Mar 24 23:53:42.084: INFO: Container busybox ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container mock ready: true, restart count 0 Mar 24 23:53:42.084: INFO: ss-0 started at 2023-03-24 23:52:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container webserver ready: true, restart count 0 Mar 24 23:53:42.084: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:42.084: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:42.084: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container coredns ready: true, restart count 0 Mar 24 23:53:42.084: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:53:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:42.084: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:53:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:42.084: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:53:23 +0000 UTC (0+7 container statuses recorded) Mar 24 23:53:42.084: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:42.084: INFO: csi-mockplugin-0 started at 2023-03-24 23:53:15 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:42.084: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:53:42.084: INFO: Container mock ready: true, restart count 0 Mar 24 23:53:42.084: INFO: hostexec-i-0713c0b421302f422-6f8db started at 2023-03-24 23:53:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.084: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:42.429: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:53:42.429: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:53:42.497: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 52703 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:53:42.497: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:53:42.578: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:53:42.664: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:42.664: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:42.664: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:53:42.664: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:42.664: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:53:42.664: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:53:42.664: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:53:42.664: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:42.664: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:53:42.664: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:53:42.664: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:53:42.664: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:42.664: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:53:42.664: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:53:42.664: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:53:42.664: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:53:42.664: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:53:42.664: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:42.664: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:53:43.017: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:53:43.017: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:53:43.093: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 62365 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-7623":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:53:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:53:39 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:34 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:34 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:34 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:53:34 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-7623^145eb68e-ca9f-11ed-99d0-3a160b8ff25d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-7623^18647fa2-ca9f-11ed-99d0-3a160b8ff25d,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-7623^145eb68e-ca9f-11ed-99d0-3a160b8ff25d,DevicePath:,},},Config:nil,},} Mar 24 23:53:43.093: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:53:43.170: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:53:43.261: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:43.261: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container coredns ready: true, restart count 0 Mar 24 23:53:43.261: INFO: inline-volume-tester2-lmsw7 started at 2023-03-24 23:53:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container csi-volume-tester ready: false, restart count 0 Mar 24 23:53:43.261: INFO: test-grpc-d881dd3a-10fc-4da5-bcc0-5415a3ac6a9a started at 2023-03-24 23:51:25 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:53:43.261: INFO: inline-volume-tester-9xsdw started at 2023-03-24 23:53:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:53:43.261: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:43.261: INFO: service-proxy-disabled-g562z started at 2023-03-24 23:53:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container service-proxy-disabled ready: true, restart count 0 Mar 24 23:53:43.261: INFO: service-proxy-toggled-2rbrj started at 2023-03-24 23:53:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container service-proxy-toggled ready: true, restart count 0 Mar 24 23:53:43.261: INFO: hostexec-i-0cdea7500a633a81f-ljdhs started at 2023-03-24 23:53:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:43.261: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:43.261: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:43.261: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:53:28 +0000 UTC (0+7 container statuses recorded) Mar 24 23:53:43.261: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:43.261: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:43.261: INFO: pod-subpath-test-preprovisionedpv-xkfd started at 2023-03-24 23:53:43 +0000 UTC (2+1 container statuses recorded) Mar 24 23:53:43.261: INFO: Init container init-volume-preprovisionedpv-xkfd ready: false, restart count 0 Mar 24 23:53:43.261: INFO: Init container test-init-volume-preprovisionedpv-xkfd ready: false, restart count 0 Mar 24 23:53:43.261: INFO: Container test-container-subpath-preprovisionedpv-xkfd ready: false, restart count 0 Mar 24 23:53:43.629: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:53:43.629 (3.119s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - dump namespaces | framework.go:209 @ 03/24/23 23:53:43.63 (3.119s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - tear down framework | framework.go:206 @ 03/24/23 23:53:43.63 STEP: Destroying namespace "volume-9856" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:53:43.63 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes - tear down framework | framework.go:206 @ 03/24/23 23:53:43.715 (86ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:53:43.716 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:53:43.716 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(ext4\)\]\svolumes\sshould\sallow\sexec\sof\sfiles\son\sthe\svolume$'
[FAILED] failed to get logs from exec-volume-test-dynamicpv-dwzn for exec-container-dynamicpv-dwzn: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-93/pods/exec-volume-test-dynamicpv-dwzn/log?container=exec-container-dynamicpv-dwzn&previous=false": stream error: stream ID 105; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:39:42.134from junit_01.xml
> Enter [BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:39:23.832 < Exit [BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/storage/framework/testsuite.go:51 @ 03/24/23 23:39:23.832 (0s) > Enter [BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes - set up framework | framework.go:191 @ 03/24/23 23:39:23.832 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:39:23.832 Mar 24 23:39:23.832: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename volume - test/e2e/framework/framework.go:250 @ 03/24/23 23:39:23.833 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:39:24.165 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:39:24.306 < Exit [BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes - set up framework | framework.go:191 @ 03/24/23 23:39:24.467 (634ms) > Enter [BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:39:24.467 < Exit [BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:39:24.467 (0s) > Enter [It] should allow exec of files on the volume - test/e2e/storage/testsuites/volumes.go:194 @ 03/24/23 23:39:24.467 Mar 24 23:39:24.467: INFO: Creating resource for dynamic PV Mar 24 23:39:24.467: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} STEP: creating a StorageClass volume-93-e2e-scgkklj - test/e2e/storage/framework/volume_resource.go:102 @ 03/24/23 23:39:24.467 STEP: creating a claim - test/e2e/storage/framework/volume_resource.go:294 @ 03/24/23 23:39:24.627 Mar 24 23:39:24.627: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil STEP: Creating pod exec-volume-test-dynamicpv-dwzn - test/e2e/storage/testsuites/volumes.go:253 @ 03/24/23 23:39:24.995 STEP: Creating a pod to test exec-volume-test - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:39:24.995 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:39:41.819 Mar 24 23:39:41.884: INFO: Trying to get logs from node i-06956c12f825d08d9 pod exec-volume-test-dynamicpv-dwzn container exec-container-dynamicpv-dwzn: <nil> Mar 24 23:39:41.951: INFO: Failed to get logs from node "i-06956c12f825d08d9" pod "exec-volume-test-dynamicpv-dwzn" container "exec-container-dynamicpv-dwzn". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-93/pods/exec-volume-test-dynamicpv-dwzn/log?container=exec-container-dynamicpv-dwzn&previous=false": stream error: stream ID 105; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:39:41.951 Mar 24 23:39:42.134: INFO: Unexpected error: <*fmt.wrapError | 0xc001092a40>: failed to get logs from exec-volume-test-dynamicpv-dwzn for exec-container-dynamicpv-dwzn: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-93/pods/exec-volume-test-dynamicpv-dwzn/log?container=exec-container-dynamicpv-dwzn&previous=false": stream error: stream ID 105; INTERNAL_ERROR; received from peer { msg: "failed to get logs from exec-volume-test-dynamicpv-dwzn for exec-container-dynamicpv-dwzn: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-93/pods/exec-volume-test-dynamicpv-dwzn/log?container=exec-container-dynamicpv-dwzn&previous=false\": stream error: stream ID 105; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc004fbb590>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-93/pods/exec-volume-test-dynamicpv-dwzn/log?container=exec-container-dynamicpv-dwzn&previous=false", Err: <http2.StreamError>{ StreamID: 105, Code: 2, Cause: <*errors.errorString | 0xc0001e7680>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from exec-volume-test-dynamicpv-dwzn for exec-container-dynamicpv-dwzn: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/volume-93/pods/exec-volume-test-dynamicpv-dwzn/log?container=exec-container-dynamicpv-dwzn&previous=false": stream error: stream ID 105; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:39:42.134 < Exit [It] should allow exec of files on the volume - test/e2e/storage/testsuites/volumes.go:194 @ 03/24/23 23:39:42.134 (17.668s) > Enter [AfterEach] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:39:42.134 Mar 24 23:39:42.134: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:39:42.214 (80ms) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/storage/testsuites/volumes.go:197 @ 03/24/23 23:39:42.214 STEP: Deleting pvc - test/e2e/storage/framework/volume_resource.go:181 @ 03/24/23 23:39:42.214 Mar 24 23:39:42.355: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comg4cwb" Mar 24 23:39:42.425: INFO: Waiting up to 5m0s for PersistentVolume pvc-c2e34a9d-3eea-4110-904a-a99903ad912e to get deleted Mar 24 23:39:42.491: INFO: PersistentVolume pvc-c2e34a9d-3eea-4110-904a-a99903ad912e found and phase=Released (65.778926ms) Mar 24 23:39:47.559: INFO: PersistentVolume pvc-c2e34a9d-3eea-4110-904a-a99903ad912e found and phase=Released (5.133717377s) Mar 24 23:39:52.627: INFO: PersistentVolume pvc-c2e34a9d-3eea-4110-904a-a99903ad912e found and phase=Released (10.20143311s) Mar 24 23:39:57.693: INFO: PersistentVolume pvc-c2e34a9d-3eea-4110-904a-a99903ad912e was removed STEP: Deleting sc - test/e2e/storage/framework/volume_resource.go:228 @ 03/24/23 23:39:57.693 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/storage/testsuites/volumes.go:197 @ 03/24/23 23:39:57.768 (15.553s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:39:57.768 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:39:57.768 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - dump namespaces | framework.go:209 @ 03/24/23 23:39:57.768 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:39:57.768 STEP: Collecting events from namespace "volume-93". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:39:57.768 STEP: Found 9 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:39:57.834 Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:24 +0000 UTC - event for ebs.csi.aws.comg4cwb: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:25 +0000 UTC - event for ebs.csi.aws.comg4cwb: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:25 +0000 UTC - event for ebs.csi.aws.comg4cwb: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } Provisioning: External provisioner is provisioning volume for claim "volume-93/ebs.csi.aws.comg4cwb" Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:29 +0000 UTC - event for ebs.csi.aws.comg4cwb: {ebs.csi.aws.com_i-0b03d8a3cb7d09376_9c2afbf7-d925-4ddf-ad66-5c7f28f9af1e } ProvisioningSucceeded: Successfully provisioned volume pvc-c2e34a9d-3eea-4110-904a-a99903ad912e Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:30 +0000 UTC - event for exec-volume-test-dynamicpv-dwzn: {default-scheduler } Scheduled: Successfully assigned volume-93/exec-volume-test-dynamicpv-dwzn to i-06956c12f825d08d9 Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:34 +0000 UTC - event for exec-volume-test-dynamicpv-dwzn: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-c2e34a9d-3eea-4110-904a-a99903ad912e" Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:38 +0000 UTC - event for exec-volume-test-dynamicpv-dwzn: {kubelet i-06956c12f825d08d9} Pulled: Container image "registry.k8s.io/e2e-test-images/nginx:1.14-4" already present on machine Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:38 +0000 UTC - event for exec-volume-test-dynamicpv-dwzn: {kubelet i-06956c12f825d08d9} Created: Created container exec-container-dynamicpv-dwzn Mar 24 23:39:57.834: INFO: At 2023-03-24 23:39:39 +0000 UTC - event for exec-volume-test-dynamicpv-dwzn: {kubelet i-06956c12f825d08d9} Started: Started container exec-container-dynamicpv-dwzn Mar 24 23:39:57.901: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:39:57.901: INFO: Mar 24 23:39:57.968: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:39:58.036: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 21672 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:39:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-03-24 23:39:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:52 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:52 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:52 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:39:52 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-00954a5e21b174818],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-00954a5e21b174818,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-00731a847fa017737,DevicePath:,},},Config:nil,},} Mar 24 23:39:58.036: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:39:58.146: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:39:58.284: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:58.284: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:39:58.284: INFO: pod-projected-configmaps-0041e20e-61be-4ff2-b71e-068a61cde1d8 started at 2023-03-24 23:38:48 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:58.284: INFO: Container createcm-volume-test ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Container delcm-volume-test ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Container updcm-volume-test ready: true, restart count 0 Mar 24 23:39:58.284: INFO: httpd started at 2023-03-24 23:39:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container httpd ready: true, restart count 0 Mar 24 23:39:58.284: INFO: test-ss-0 started at 2023-03-24 23:39:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container webserver ready: true, restart count 0 Mar 24 23:39:58.284: INFO: startup-8fdbdc1a-89d6-4268-84b7-a6654ac09481 started at 2023-03-24 23:36:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container busybox ready: false, restart count 0 Mar 24 23:39:58.284: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:39:58.284: INFO: pod-ephm-test-projected-6sb7 started at 2023-03-24 23:39:57 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container test-container-subpath-projected-6sb7 ready: false, restart count 0 Mar 24 23:39:58.284: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:39:58.284: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:39:58.284: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:39:58.284: INFO: busybox-c09a66e2-3e82-43de-8fb7-6403b63c1ccc started at 2023-03-24 23:36:37 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container busybox ready: true, restart count 0 Mar 24 23:39:58.284: INFO: ss-0 started at 2023-03-24 23:39:53 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.284: INFO: Container webserver ready: false, restart count 0 Mar 24 23:39:58.579: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:39:58.579: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:39:58.646: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 21535 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-7537":"i-06956c12f825d08d9","ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:39:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:54 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:54 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:54 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:39:54 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:39:58.646: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:39:58.740: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:39:58.814: INFO: busybox-scheduling-a49d94ac-8e54-421a-8cbb-3f5230dd2585 started at 2023-03-24 23:38:50 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.814: INFO: Container busybox-scheduling-a49d94ac-8e54-421a-8cbb-3f5230dd2585 ready: true, restart count 0 Mar 24 23:39:58.814: INFO: hostexec-i-06956c12f825d08d9-sh4kw started at 2023-03-24 23:39:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.814: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:39:58.814: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:58.814: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:39:58.814: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.814: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:39:58.814: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:39:58.814: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:39:58.814: INFO: local-client started at 2023-03-24 23:39:50 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.814: INFO: Container local-client ready: true, restart count 0 Mar 24 23:39:58.814: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:39:30 +0000 UTC (0+7 container statuses recorded) Mar 24 23:39:58.814: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:39:58.814: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:39:58.814: INFO: run-test started at 2023-03-24 23:39:49 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:58.814: INFO: Container run-test ready: false, restart count 0 Mar 24 23:39:59.099: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:39:59.099: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:39:59.166: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 21510 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:39:53 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:39:53 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0b62da9cfc622608f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0b62da9cfc622608f,DevicePath:,},},Config:nil,},} Mar 24 23:39:59.167: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:39:59.244: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:39:59.316: INFO: csi-mockplugin-0 started at 2023-03-24 23:39:55 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:59.316: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container mock ready: true, restart count 0 Mar 24 23:39:59.316: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Container coredns ready: true, restart count 0 Mar 24 23:39:59.316: INFO: pod-subpath-test-preprovisionedpv-zf7k started at 2023-03-24 23:39:13 +0000 UTC (1+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Init container init-volume-preprovisionedpv-zf7k ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container test-container-subpath-preprovisionedpv-zf7k ready: false, restart count 0 Mar 24 23:39:59.316: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:39:56 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:39:59.316: INFO: pod-3df6a078-383a-4caf-bbe2-202c45fccbc6 started at 2023-03-24 23:39:51 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Container write-pod ready: false, restart count 0 Mar 24 23:39:59.316: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:39:59.316: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:39:59.316: INFO: dns-test-15417daf-e644-4f8b-957d-19e7a8dd8ad8 started at 2023-03-24 23:39:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:59.316: INFO: Container jessie-querier ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container querier ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container webserver ready: true, restart count 0 Mar 24 23:39:59.316: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:59.316: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:39:59.316: INFO: hostexec-i-0713c0b421302f422-p2nb8 started at 2023-03-24 23:38:57 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.316: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:39:59.316: INFO: dns-test-07d9f2c6-2f98-41bc-acda-889649d2b3c0 started at 2023-03-24 23:39:45 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:59.316: INFO: Container jessie-querier ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container querier ready: true, restart count 0 Mar 24 23:39:59.316: INFO: Container webserver ready: true, restart count 0 Mar 24 23:39:59.611: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:39:59.611: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:39:59.677: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 5810 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:35:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:35:08 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:39:59.678: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:39:59.752: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:39:59.824: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:39:59.824: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:39:59.824: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:39:59.824: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:39:59.824: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:39:59.824: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:39:59.824: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:39:59.824: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:39:59.824: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:39:59.824: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:39:59.824: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:39:59.824: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:39:59.824: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:39:59.824: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:39:59.824: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:39:59.824: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:39:59.824: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:39:59.824: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:39:59.824: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:40:00.156: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:40:00.156: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:40:00.225: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 21598 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6067":"i-0cdea7500a633a81f","csi-mock-csi-mock-volumes-expansion-3855":"i-0cdea7500a633a81f","csi-mock-csi-mock-volumes-service-token-9084":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:39:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:39:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:56 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:56 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:39:56 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:39:56 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-6067^24f3bfa1-ca9d-11ed-817d-7a905b676431 kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b kubernetes.io/csi/csi-mock-csi-mock-volumes-service-token-9084^29a83733-ca9d-11ed-bc1b-6e7e0820b488],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6067^24f3bfa1-ca9d-11ed-817d-7a905b676431,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-3855^051a4d90-ca9d-11ed-a705-16ddc87a996b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-service-token-9084^29a83733-ca9d-11ed-bc1b-6e7e0820b488,DevicePath:,},},Config:nil,},} Mar 24 23:40:00.226: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:40:00.321: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:40:00.407: INFO: csi-mockplugin-0 started at 2023-03-24 23:38:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:00.407: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container mock ready: true, restart count 0 Mar 24 23:40:00.407: INFO: pvc-volume-tester-vgkcq started at 2023-03-24 23:38:46 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:40:00.407: INFO: pod-cb4de73d-db51-4306-a7db-c514c3aba02e started at 2023-03-24 23:39:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container write-pod ready: true, restart count 0 Mar 24 23:40:00.407: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:39:34 +0000 UTC (0+7 container statuses recorded) Mar 24 23:40:00.407: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:00.407: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:40:00.407: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:40:00.407: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:00.407: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:00.407: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:39:42 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:40:00.407: INFO: pvc-volume-tester-h89gk started at 2023-03-24 23:39:47 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container volume-tester ready: true, restart count 0 Mar 24 23:40:00.407: INFO: hostexec-i-0cdea7500a633a81f-8q2nq started at 2023-03-24 23:39:54 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:40:00.407: INFO: csi-mockplugin-0 started at 2023-03-24 23:38:56 +0000 UTC (0+4 container statuses recorded) Mar 24 23:40:00.407: INFO: Container busybox ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container mock ready: true, restart count 0 Mar 24 23:40:00.407: INFO: pod-8e115740-bb65-41ff-9bfc-e63f8b33d8da started at 2023-03-24 23:39:58 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container write-pod ready: true, restart count 0 Mar 24 23:40:00.407: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:40:00.407: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container coredns ready: true, restart count 0 Mar 24 23:40:00.407: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:38:40 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:00.407: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:40:00.407: INFO: csi-mockplugin-0 started at 2023-03-24 23:39:42 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:00.407: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:40:00.407: INFO: Container mock ready: true, restart count 0 Mar 24 23:40:00.732: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:40:00.732 (2.964s) < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - dump namespaces | framework.go:209 @ 03/24/23 23:40:00.732 (2.965s) > Enter [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - tear down framework | framework.go:206 @ 03/24/23 23:40:00.732 STEP: Destroying namespace "volume-93" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:40:00.732 < Exit [DeferCleanup (Each)] [Testpattern: Dynamic PV (ext4)] volumes - tear down framework | framework.go:206 @ 03/24/23 23:40:00.802 (70ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:40:00.802 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:40:00.803 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-auth\]\sServiceAccounts\sServiceAccountIssuerDiscovery\sshould\ssupport\sOIDC\sdiscovery\sof\sservice\saccount\sissuer\s\[Conformance\]$'
[FAILED] Unexpected error getting pod logs: timed out waiting for the condition In [It] at: test/e2e/auth/service_accounts.go:630 @ 03/24/23 23:53:16.693from junit_01.xml
> Enter [BeforeEach] [sig-auth] ServiceAccounts - set up framework | framework.go:191 @ 03/24/23 23:51:11.631 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:51:11.631 Mar 24 23:51:11.631: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename svcaccounts - test/e2e/framework/framework.go:250 @ 03/24/23 23:51:11.632 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:51:11.835 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:51:11.966 < Exit [BeforeEach] [sig-auth] ServiceAccounts - set up framework | framework.go:191 @ 03/24/23 23:51:12.105 (474ms) > Enter [BeforeEach] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:51:12.105 < Exit [BeforeEach] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:51:12.105 (0s) > Enter [It] ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance] - test/e2e/auth/service_accounts.go:529 @ 03/24/23 23:51:12.105 Mar 24 23:51:12.270: INFO: created pod STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:51:16.488 Mar 24 23:51:46.488: INFO: polling logs Mar 24 23:51:46.565: INFO: Error pulling logs: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-1116/pods/oidc-discovery-validator/log?container=oidc-discovery-validator&previous=false": stream error: stream ID 3517; INTERNAL_ERROR; received from peer Mar 24 23:52:16.488: INFO: polling logs Mar 24 23:52:16.555: INFO: Error pulling logs: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-1116/pods/oidc-discovery-validator/log?container=oidc-discovery-validator&previous=false": stream error: stream ID 3519; INTERNAL_ERROR; received from peer Mar 24 23:52:46.488: INFO: polling logs Mar 24 23:52:46.557: INFO: Error pulling logs: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-1116/pods/oidc-discovery-validator/log?container=oidc-discovery-validator&previous=false": stream error: stream ID 3521; INTERNAL_ERROR; received from peer Mar 24 23:53:16.489: INFO: polling logs Mar 24 23:53:16.556: INFO: Error pulling logs: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-1116/pods/oidc-discovery-validator/log?container=oidc-discovery-validator&previous=false": stream error: stream ID 3523; INTERNAL_ERROR; received from peer Mar 24 23:53:16.556: INFO: polling logs Mar 24 23:53:16.624: INFO: Error pulling logs: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-1116/pods/oidc-discovery-validator/log?container=oidc-discovery-validator&previous=false": stream error: stream ID 3525; INTERNAL_ERROR; received from peer [FAILED] Unexpected error getting pod logs: timed out waiting for the condition In [It] at: test/e2e/auth/service_accounts.go:630 @ 03/24/23 23:53:16.693 < Exit [It] ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance] - test/e2e/auth/service_accounts.go:529 @ 03/24/23 23:53:16.693 (2m4.589s) > Enter [AfterEach] [sig-auth] ServiceAccounts - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:53:16.693 Mar 24 23:53:16.693: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-auth] ServiceAccounts - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:53:16.762 (69ms) > Enter [DeferCleanup (Each)] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:53:16.762 < Exit [DeferCleanup (Each)] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:53:16.762 (0s) > Enter [DeferCleanup (Each)] [sig-auth] ServiceAccounts - dump namespaces | framework.go:209 @ 03/24/23 23:53:16.762 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:53:16.762 STEP: Collecting events from namespace "svcaccounts-1116". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:53:16.762 STEP: Found 4 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:53:16.828 Mar 24 23:53:16.828: INFO: At 2023-03-24 23:51:12 +0000 UTC - event for oidc-discovery-validator: {default-scheduler } Scheduled: Successfully assigned svcaccounts-1116/oidc-discovery-validator to i-0cdea7500a633a81f Mar 24 23:53:16.829: INFO: At 2023-03-24 23:51:13 +0000 UTC - event for oidc-discovery-validator: {kubelet i-0cdea7500a633a81f} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:53:16.829: INFO: At 2023-03-24 23:51:13 +0000 UTC - event for oidc-discovery-validator: {kubelet i-0cdea7500a633a81f} Created: Created container oidc-discovery-validator Mar 24 23:53:16.829: INFO: At 2023-03-24 23:51:13 +0000 UTC - event for oidc-discovery-validator: {kubelet i-0cdea7500a633a81f} Started: Started container oidc-discovery-validator Mar 24 23:53:16.895: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:53:16.895: INFO: oidc-discovery-validator i-0cdea7500a633a81f Succeeded [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-24 23:51:12 +0000 UTC PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-03-24 23:51:14 +0000 UTC PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-03-24 23:51:14 +0000 UTC PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-24 23:51:12 +0000 UTC }] Mar 24 23:53:16.895: INFO: Mar 24 23:53:16.962: INFO: Unable to fetch svcaccounts-1116/oidc-discovery-validator/oidc-discovery-validator logs: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-1116/pods/oidc-discovery-validator/log?container=oidc-discovery-validator&previous=false&tailLines=42": stream error: stream ID 3535; INTERNAL_ERROR; received from peer Mar 24 23:53:17.029: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:53:17.096: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 60865 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:53:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:52:57 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-080957e38bee7cecb],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:53:17.097: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:53:17.171: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:53:17.246: INFO: service-proxy-disabled-6nspv started at 2023-03-24 23:53:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container service-proxy-disabled ready: true, restart count 0 Mar 24 23:53:17.246: INFO: execpod-rtd42 started at 2023-03-24 23:53:08 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:17.246: INFO: webserver-pod started at 2023-03-24 23:51:23 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container agnhost ready: false, restart count 0 Mar 24 23:53:17.246: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:17.246: INFO: webserver-pod started at 2023-03-24 23:53:16 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container agnhost-container ready: false, restart count 0 Mar 24 23:53:17.246: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:17.246: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:17.246: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:17.246: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:17.246: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:53:17.246: INFO: test-ss-0 started at 2023-03-24 23:53:08 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container webserver ready: true, restart count 0 Mar 24 23:53:17.246: INFO: service-proxy-toggled-25qww started at 2023-03-24 23:53:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container service-proxy-toggled ready: true, restart count 0 Mar 24 23:53:17.246: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:17.246: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:17.246: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:17.246: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:17.246: INFO: ss-2 started at 2023-03-24 23:52:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.246: INFO: Container webserver ready: false, restart count 0 Mar 24 23:53:17.580: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:53:17.580: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:53:17.647: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 61096 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:53:09 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:53:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:53:09 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0240efc31331fc483],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0240efc31331fc483,DevicePath:,},},Config:nil,},} Mar 24 23:53:17.648: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:53:17.724: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:53:17.802: INFO: service-proxy-disabled-49qrd started at 2023-03-24 23:53:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Container service-proxy-disabled ready: true, restart count 0 Mar 24 23:53:17.802: INFO: pod-csi-inline-volumes started at 2023-03-24 23:52:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 24 23:53:17.802: INFO: pod-configmaps-267d1d9e-c0fc-45e7-b918-870e2680fc24 started at 2023-03-24 23:50:20 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:17.802: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:17.802: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:17.802: INFO: pod-projected-secrets-f9adb492-b0f3-484a-9832-f9db7c2dca6f started at 2023-03-24 23:49:19 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:17.802: INFO: Container creates-volume-test ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container dels-volume-test ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container upds-volume-test ready: true, restart count 0 Mar 24 23:53:17.802: INFO: test-pod started at 2023-03-24 23:53:01 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:17.802: INFO: Container busybox-1 ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container busybox-2 ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container busybox-3 ready: true, restart count 0 Mar 24 23:53:17.802: INFO: service-proxy-toggled-sh662 started at 2023-03-24 23:53:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Container service-proxy-toggled ready: true, restart count 0 Mar 24 23:53:17.802: INFO: pod-configmaps-5e207db0-ceab-4dc0-81c4-130e29bd58ba started at 2023-03-24 23:49:48 +0000 UTC (0+2 container statuses recorded) Mar 24 23:53:17.802: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container configmap-volume-binary-test ready: false, restart count 0 Mar 24 23:53:17.802: INFO: ss-1 started at 2023-03-24 23:52:25 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:17.802: INFO: Container webserver ready: true, restart count 0 Mar 24 23:53:17.802: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:17.802: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:17.802: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:18.171: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:53:18.171: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:53:18.247: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 60873 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:53:00 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:53:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:00 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:00 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:53:00 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:53:00 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-00477843dedaa7dfd kubernetes.io/csi/ebs.csi.aws.com^vol-02182534a84d7fe87 kubernetes.io/csi/ebs.csi.aws.com^vol-02e4eaa794a49d709],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-00477843dedaa7dfd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02e4eaa794a49d709,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02182534a84d7fe87,DevicePath:,},},Config:nil,},} Mar 24 23:53:18.248: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:53:18.352: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:53:18.426: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:18.426: INFO: csi-mockplugin-0 started at 2023-03-24 23:53:13 +0000 UTC (0+4 container statuses recorded) Mar 24 23:53:18.426: INFO: Container busybox ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container mock ready: true, restart count 0 Mar 24 23:53:18.426: INFO: ss-0 started at 2023-03-24 23:52:10 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Container webserver ready: true, restart count 0 Mar 24 23:53:18.426: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:18.426: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:18.426: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Container coredns ready: true, restart count 0 Mar 24 23:53:18.426: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:53:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:18.426: INFO: csi-mockplugin-resizer-0 started at 2023-03-24 23:53:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:18.426: INFO: pod-subpath-test-inlinevolume-gthf started at 2023-03-24 23:53:00 +0000 UTC (1+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Init container init-volume-inlinevolume-gthf ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container test-container-subpath-inlinevolume-gthf ready: true, restart count 0 Mar 24 23:53:18.426: INFO: pod-subpath-test-preprovisionedpv-j78k started at 2023-03-24 23:53:14 +0000 UTC (1+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Init container init-volume-preprovisionedpv-j78k ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container test-container-subpath-preprovisionedpv-j78k ready: true, restart count 0 Mar 24 23:53:18.426: INFO: csi-mockplugin-0 started at 2023-03-24 23:53:15 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:18.426: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container driver-registrar ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container mock ready: true, restart count 0 Mar 24 23:53:18.426: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:18.426: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:18.426: INFO: hostexec-i-0713c0b421302f422-ffsxq started at 2023-03-24 23:52:56 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.426: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:18.735: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:53:18.735: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:53:18.807: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 52703 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:50:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:50:27 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/kube-scheduler@sha256:cbf4007abb076188f16b975bc2b74452603cc7b34c60a29d92f14c38ad02dc11 registry.k8s.io/kube-scheduler:v1.28.0-alpha.0],SizeBytes:18078674,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:5074298,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:53:18.807: INFO: Logging kubelet events for node i-0b03d8a3cb7d09376 Mar 24 23:53:18.889: INFO: Logging pods the kubelet thinks is on node i-0b03d8a3cb7d09376 Mar 24 23:53:18.965: INFO: ebs-csi-node-89jrh started at 2023-03-24 23:17:57 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:18.965: INFO: Container ebs-plugin ready: true, restart count 2 Mar 24 23:53:18.965: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:18.965: INFO: dns-controller-69987775c6-6kvfs started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container dns-controller ready: true, restart count 0 Mar 24 23:53:18.965: INFO: kops-controller-fx5rq started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container kops-controller ready: true, restart count 0 Mar 24 23:53:18.965: INFO: kube-controller-manager-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container kube-controller-manager ready: true, restart count 4 Mar 24 23:53:18.965: INFO: kube-proxy-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:18.965: INFO: kube-scheduler-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container kube-scheduler ready: true, restart count 0 Mar 24 23:53:18.965: INFO: etcd-manager-main-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:53:18.965: INFO: calico-node-pnvdm started at 2023-03-24 23:17:57 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:18.965: INFO: ebs-csi-controller-79f984545f-g52vq started at 2023-03-24 23:19:17 +0000 UTC (0+5 container statuses recorded) Mar 24 23:53:18.965: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:18.965: INFO: kube-apiserver-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (0+2 container statuses recorded) Mar 24 23:53:18.965: INFO: Container healthcheck ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container kube-apiserver ready: true, restart count 2 Mar 24 23:53:18.965: INFO: etcd-manager-events-i-0b03d8a3cb7d09376 started at 2023-03-24 23:15:43 +0000 UTC (11+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Init container init-etcd-3-5-7 ready: true, restart count 0 Mar 24 23:53:18.965: INFO: Container etcd-manager ready: true, restart count 0 Mar 24 23:53:18.965: INFO: aws-cloud-controller-manager-ss4bn started at 2023-03-24 23:18:06 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Mar 24 23:53:18.965: INFO: calico-kube-controllers-55f488656-7h4s8 started at 2023-03-24 23:18:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:18.965: INFO: Container calico-kube-controllers ready: true, restart count 0 Mar 24 23:53:19.319: INFO: Latency metrics for node i-0b03d8a3cb7d09376 Mar 24 23:53:19.319: INFO: Logging node info for node i-0cdea7500a633a81f Mar 24 23:53:19.386: INFO: Node Info: &Node{ObjectMeta:{i-0cdea7500a633a81f 4ed2afbf-9235-4861-bb12-8e6a4138d822 60001 0 2023-03-24 23:29:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0cdea7500a633a81f kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0cdea7500a633a81f topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2358":"i-0cdea7500a633a81f","ebs.csi.aws.com":"i-0cdea7500a633a81f"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.42.71/19 projectcalico.org/IPv4IPIPTunnelAddr:100.113.209.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:29:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:29:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:29:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.8.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:29:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:52:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:52:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.8.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0cdea7500a633a81f,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.8.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:29:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:53 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:52:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:52:53 +0000 UTC,LastTransitionTime:2023-03-24 23:29:44 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.42.71,},NodeAddress{Type:ExternalIP,Address:18.246.0.23,},NodeAddress{Type:InternalDNS,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0cdea7500a633a81f.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-246-0-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bd0e70bb6d2fbc297134bc31553,SystemUUID:ec2f5bd0-e70b-b6d2-fbc2-97134bc31553,BootID:bac83555-6a73-426c-b517-cbfb00a3f582,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-2358^f9bc9d25-ca9e-11ed-80b7-46d69743faa0],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-2358^f9bc9d25-ca9e-11ed-80b7-46d69743faa0,DevicePath:,},},Config:nil,},} Mar 24 23:53:19.386: INFO: Logging kubelet events for node i-0cdea7500a633a81f Mar 24 23:53:19.466: INFO: Logging pods the kubelet thinks is on node i-0cdea7500a633a81f Mar 24 23:53:19.539: INFO: calico-node-bhsgg started at 2023-03-24 23:29:34 +0000 UTC (3+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:53:19.539: INFO: coredns-559769c974-6mjjp started at 2023-03-24 23:34:32 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container coredns ready: true, restart count 0 Mar 24 23:53:19.539: INFO: oidc-discovery-validator started at 2023-03-24 23:51:12 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container oidc-discovery-validator ready: false, restart count 0 Mar 24 23:53:19.539: INFO: test-grpc-d881dd3a-10fc-4da5-bcc0-5415a3ac6a9a started at 2023-03-24 23:51:25 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container agnhost ready: true, restart count 0 Mar 24 23:53:19.539: INFO: kube-proxy-i-0cdea7500a633a81f started at 2023-03-24 23:29:34 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:53:19.539: INFO: service-proxy-disabled-g562z started at 2023-03-24 23:53:04 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container service-proxy-disabled ready: true, restart count 0 Mar 24 23:53:19.539: INFO: service-proxy-toggled-2rbrj started at 2023-03-24 23:53:07 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container service-proxy-toggled ready: true, restart count 0 Mar 24 23:53:19.539: INFO: ebs-csi-node-6mwpv started at 2023-03-24 23:29:34 +0000 UTC (0+3 container statuses recorded) Mar 24 23:53:19.539: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:19.539: INFO: hostexec-i-0cdea7500a633a81f-mtchk started at 2023-03-24 23:53:11 +0000 UTC (0+1 container statuses recorded) Mar 24 23:53:19.539: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:53:19.539: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:52:43 +0000 UTC (0+7 container statuses recorded) Mar 24 23:53:19.539: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:53:19.539: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:53:19.853: INFO: Latency metrics for node i-0cdea7500a633a81f END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:53:19.853 (3.091s) < Exit [DeferCleanup (Each)] [sig-auth] ServiceAccounts - dump namespaces | framework.go:209 @ 03/24/23 23:53:19.853 (3.091s) > Enter [DeferCleanup (Each)] [sig-auth] ServiceAccounts - tear down framework | framework.go:206 @ 03/24/23 23:53:19.853 STEP: Destroying namespace "svcaccounts-1116" for this suite. - test/e2e/framework/framework.go:351 @ 03/24/23 23:53:19.853 < Exit [DeferCleanup (Each)] [sig-auth] ServiceAccounts - tear down framework | framework.go:206 @ 03/24/23 23:53:19.935 (81ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:53:19.935 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/24/23 23:53:19.935 (0s)
Find logs mentions in log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-auth\]\sServiceAccounts\sshould\smount\sprojected\sservice\saccount\stoken\s\[Conformance\]$'
[FAILED] failed to get logs from test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad for agnhost-container: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-3947/pods/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad/log?container=agnhost-container&previous=false": stream error: stream ID 1053; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:40:38.372
> Enter [BeforeEach] [sig-auth] ServiceAccounts - set up framework | framework.go:191 @ 03/24/23 23:40:33.299 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/24/23 23:40:33.299 Mar 24 23:40:33.299: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename svcaccounts - test/e2e/framework/framework.go:250 @ 03/24/23 23:40:33.3 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/24/23 23:40:33.497 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/24/23 23:40:33.627 < Exit [BeforeEach] [sig-auth] ServiceAccounts - set up framework | framework.go:191 @ 03/24/23 23:40:33.757 (458ms) > Enter [BeforeEach] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:40:33.757 < Exit [BeforeEach] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:33 @ 03/24/23 23:40:33.757 (0s) > Enter [It] should mount projected service account token [Conformance] - test/e2e/auth/service_accounts.go:275 @ 03/24/23 23:40:33.757 STEP: Creating a pod to test service account token: - test/e2e/framework/pod/output/output.go:243 @ 03/24/23 23:40:33.757 STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/24/23 23:40:38.042 Mar 24 23:40:38.108: INFO: Trying to get logs from node i-0cdea7500a633a81f pod test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad container agnhost-container: <nil> Mar 24 23:40:38.179: INFO: Failed to get logs from node "i-0cdea7500a633a81f" pod "test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad" container "agnhost-container". Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-3947/pods/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad/log?container=agnhost-container&previous=false": stream error: stream ID 1053; INTERNAL_ERROR; received from peer STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/24/23 23:40:38.179 Mar 24 23:40:38.371: INFO: Unexpected error: <*fmt.wrapError | 0xc003a280a0>: failed to get logs from test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad for agnhost-container: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-3947/pods/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad/log?container=agnhost-container&previous=false": stream error: stream ID 1053; INTERNAL_ERROR; received from peer { msg: "failed to get logs from test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad for agnhost-container: Get \"https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-3947/pods/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad/log?container=agnhost-container&previous=false\": stream error: stream ID 1053; INTERNAL_ERROR; received from peer", err: <*url.Error | 0xc002f23a40>{ Op: "Get", URL: "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-3947/pods/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad/log?container=agnhost-container&previous=false", Err: <http2.StreamError>{ StreamID: 1053, Code: 2, Cause: <*errors.errorString | 0xc000233650>{ s: "received from peer", }, }, }, } [FAILED] failed to get logs from test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad for agnhost-container: Get "https://api.e2e-3a5daf4e9a-96255.test-cncf-aws.k8s.io/api/v1/namespaces/svcaccounts-3947/pods/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad/log?container=agnhost-container&previous=false": stream error: stream ID 1053; INTERNAL_ERROR; received from peer In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/24/23 23:40:38.372 < Exit [It] should mount projected service account token [Conformance] - test/e2e/auth/service_accounts.go:275 @ 03/24/23 23:40:38.372 (4.615s) > Enter [AfterEach] [sig-auth] ServiceAccounts - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:40:38.372 Mar 24 23:40:38.372: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-auth] ServiceAccounts - test/e2e/framework/node/init/init.go:33 @ 03/24/23 23:40:38.438 (67ms) > Enter [DeferCleanup (Each)] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:40:38.438 < Exit [DeferCleanup (Each)] [sig-auth] ServiceAccounts - test/e2e/framework/metrics/init/init.go:35 @ 03/24/23 23:40:38.439 (0s) > Enter [DeferCleanup (Each)] [sig-auth] ServiceAccounts - dump namespaces | framework.go:209 @ 03/24/23 23:40:38.439 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/24/23 23:40:38.439 STEP: Collecting events from namespace "svcaccounts-3947". - test/e2e/framework/debug/dump.go:42 @ 03/24/23 23:40:38.439 STEP: Found 4 events. - test/e2e/framework/debug/dump.go:46 @ 03/24/23 23:40:38.504 Mar 24 23:40:38.504: INFO: At 2023-03-24 23:40:33 +0000 UTC - event for test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad: {default-scheduler } Scheduled: Successfully assigned svcaccounts-3947/test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad to i-0cdea7500a633a81f Mar 24 23:40:38.504: INFO: At 2023-03-24 23:40:34 +0000 UTC - event for test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad: {kubelet i-0cdea7500a633a81f} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 24 23:40:38.504: INFO: At 2023-03-24 23:40:34 +0000 UTC - event for test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad: {kubelet i-0cdea7500a633a81f} Created: Created container agnhost-container Mar 24 23:40:38.504: INFO: At 2023-03-24 23:40:34 +0000 UTC - event for test-pod-ad5bc7eb-1ddc-43b9-af8b-38909ecc73ad: {kubelet i-0cdea7500a633a81f} Started: Started container agnhost-container Mar 24 23:40:38.568: INFO: POD NODE PHASE GRACE CONDITIONS Mar 24 23:40:38.568: INFO: Mar 24 23:40:38.634: INFO: Logging node info for node i-005794773d640afd9 Mar 24 23:40:38.704: INFO: Node Info: &Node{ObjectMeta:{i-005794773d640afd9 0e8862b0-5477-47c1-8a61-5e2ebb00d3a7 23514 0 2023-03-24 23:21:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-005794773d640afd9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-005794773d640afd9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4593":"i-005794773d640afd9","csi-hostpath-ephemeral-7123":"i-005794773d640afd9","ebs.csi.aws.com":"i-005794773d640afd9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.36.199/19 projectcalico.org/IPv4IPIPTunnelAddr:100.96.102.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:21:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:21:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.6.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:21:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:40:33 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:40:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.6.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-005794773d640afd9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.6.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:21:58 +0000 UTC,LastTransitionTime:2023-03-24 23:21:58 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:33 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:33 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:33 +0000 UTC,LastTransitionTime:2023-03-24 23:21:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:33 +0000 UTC,LastTransitionTime:2023-03-24 23:21:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.36.199,},NodeAddress{Type:ExternalIP,Address:44.242.142.48,},NodeAddress{Type:InternalDNS,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-005794773d640afd9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-44-242-142-48.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2795659027bcaf94f7c9b3d5fef0a9,SystemUUID:ec279565-9027-bcaf-94f7-c9b3d5fef0a9,BootID:7520dc58-0cb9-43df-8346-35f34d411aba,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-07639ee71474d107a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-07639ee71474d107a,DevicePath:,},},Config:nil,},} Mar 24 23:40:38.704: INFO: Logging kubelet events for node i-005794773d640afd9 Mar 24 23:40:38.780: INFO: Logging pods the kubelet thinks is on node i-005794773d640afd9 Mar 24 23:40:38.856: INFO: pod-ephm-test-projected-6sb7 started at 2023-03-24 23:39:57 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container test-container-subpath-projected-6sb7 ready: false, restart count 0 Mar 24 23:40:38.856: INFO: kube-proxy-i-005794773d640afd9 started at 2023-03-24 23:21:39 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:40:38.856: INFO: coredns-autoscaler-7cb5c5b969-hr5gb started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container autoscaler ready: true, restart count 0 Mar 24 23:40:38.856: INFO: busybox-c09a66e2-3e82-43de-8fb7-6403b63c1ccc started at 2023-03-24 23:36:37 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container busybox ready: true, restart count 0 Mar 24 23:40:38.856: INFO: inline-volume-tester-p4nf7 started at 2023-03-24 23:40:22 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container csi-volume-tester ready: true, restart count 0 Mar 24 23:40:38.856: INFO: calico-node-gbnp9 started at 2023-03-24 23:21:40 +0000 UTC (3+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:40:38.856: INFO: pod-projected-configmaps-0041e20e-61be-4ff2-b71e-068a61cde1d8 started at 2023-03-24 23:38:48 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:38.856: INFO: Container createcm-volume-test ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container delcm-volume-test ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container updcm-volume-test ready: true, restart count 0 Mar 24 23:40:38.856: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:40:22 +0000 UTC (0+7 container statuses recorded) Mar 24 23:40:38.856: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:38.856: INFO: ebs-csi-node-mb8tl started at 2023-03-24 23:21:40 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:38.856: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:38.856: INFO: test-ss-0 started at 2023-03-24 23:39:52 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container webserver ready: true, restart count 0 Mar 24 23:40:38.856: INFO: csi-hostpathplugin-0 started at 2023-03-24 23:40:35 +0000 UTC (0+7 container statuses recorded) Mar 24 23:40:38.856: INFO: Container csi-attacher ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container csi-provisioner ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container csi-resizer ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container csi-snapshotter ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container hostpath ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:38.856: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:38.856: INFO: inline-volume-tester-l8p2j started at 2023-03-24 23:40:38 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:38.856: INFO: Container csi-volume-tester ready: false, restart count 0 Mar 24 23:40:39.182: INFO: Latency metrics for node i-005794773d640afd9 Mar 24 23:40:39.182: INFO: Logging node info for node i-06956c12f825d08d9 Mar 24 23:40:39.247: INFO: Node Info: &Node{ObjectMeta:{i-06956c12f825d08d9 773e43a5-46dc-4dbb-8367-139c5d257c88 23423 0 2023-03-24 23:33:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06956c12f825d08d9 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-06956c12f825d08d9 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06956c12f825d08d9"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.51.145/19 projectcalico.org/IPv4IPIPTunnelAddr:100.105.123.192 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:33:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:33:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:33:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.9.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:33:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:40:17 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-24 23:40:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.9.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-06956c12f825d08d9,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.9.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:33:43 +0000 UTC,LastTransitionTime:2023-03-24 23:33:43 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:35 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:35 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:35 +0000 UTC,LastTransitionTime:2023-03-24 23:33:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:35 +0000 UTC,LastTransitionTime:2023-03-24 23:33:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.145,},NodeAddress{Type:ExternalIP,Address:35.85.232.165,},NodeAddress{Type:InternalDNS,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06956c12f825d08d9.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-85-232-165.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2ce7c84bf1acf218bd5da3418f8e54,SystemUUID:ec2ce7c8-4bf1-acf2-18bd-5da3418f8e54,BootID:e4627f31-bb9c-4472-adeb-418046bec783,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-05ae9e37ff4774dd9 kubernetes.io/csi/ebs.csi.aws.com^vol-0b0df699051d9fc84],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0b0df699051d9fc84,DevicePath:,},},Config:nil,},} Mar 24 23:40:39.248: INFO: Logging kubelet events for node i-06956c12f825d08d9 Mar 24 23:40:39.328: INFO: Logging pods the kubelet thinks is on node i-06956c12f825d08d9 Mar 24 23:40:39.419: INFO: concurrent-27995020-fmx54 started at 2023-03-24 23:40:00 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Container c ready: false, restart count 0 Mar 24 23:40:39.419: INFO: pvc-volume-tester-writer-dhc6k started at 2023-03-24 23:40:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Container volume-tester ready: false, restart count 0 Mar 24 23:40:39.419: INFO: ebs-csi-node-scjg2 started at 2023-03-24 23:33:26 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:39.419: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:39.419: INFO: send-events-906538f8-a5d3-43ac-a764-3d70483cf21b started at 2023-03-24 23:40:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Container p ready: true, restart count 0 Mar 24 23:40:39.419: INFO: ss-1 started at 2023-03-24 23:40:15 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Container webserver ready: false, restart count 0 Mar 24 23:40:39.419: INFO: test-rs-h2gl4 started at 2023-03-24 23:40:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Container httpd ready: true, restart count 0 Mar 24 23:40:39.419: INFO: pod-subpath-test-configmap-tv2d started at 2023-03-24 23:40:35 +0000 UTC (1+2 container statuses recorded) Mar 24 23:40:39.419: INFO: Init container init-volume-configmap-tv2d ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Container test-container-subpath-configmap-tv2d ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Container test-container-volume-configmap-tv2d ready: true, restart count 0 Mar 24 23:40:39.419: INFO: kube-proxy-i-06956c12f825d08d9 started at 2023-03-24 23:33:26 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:40:39.419: INFO: calico-node-9w5vv started at 2023-03-24 23:33:26 +0000 UTC (3+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:40:39.419: INFO: pod-subpath-test-inlinevolume-xntc started at 2023-03-24 23:40:23 +0000 UTC (1+1 container statuses recorded) Mar 24 23:40:39.419: INFO: Init container init-volume-inlinevolume-xntc ready: true, restart count 0 Mar 24 23:40:39.419: INFO: Container test-container-subpath-inlinevolume-xntc ready: true, restart count 0 Mar 24 23:40:39.797: INFO: Latency metrics for node i-06956c12f825d08d9 Mar 24 23:40:39.797: INFO: Logging node info for node i-0713c0b421302f422 Mar 24 23:40:39.864: INFO: Node Info: &Node{ObjectMeta:{i-0713c0b421302f422 7ca94c8e-1815-4de3-b79f-e5dd8b105e46 23677 0 2023-03-24 23:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0713c0b421302f422 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2d topology.hostpath.csi/node:i-0713c0b421302f422 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0713c0b421302f422"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.53.181/19 projectcalico.org/IPv4IPIPTunnelAddr:100.98.61.0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-03-24 23:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-03-24 23:25:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.7.0/24\"":{}}}} } {calico-node Update v1 2023-03-24 23:25:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:40:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.7.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0713c0b421302f422,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.7.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:25:54 +0000 UTC,LastTransitionTime:2023-03-24 23:25:54 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:34 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:34 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:34 +0000 UTC,LastTransitionTime:2023-03-24 23:25:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:34 +0000 UTC,LastTransitionTime:2023-03-24 23:25:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.181,},NodeAddress{Type:ExternalIP,Address:35.88.5.23,},NodeAddress{Type:InternalDNS,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0713c0b421302f422.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-88-5-23.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec259c8336b5f568cf65ad8234e187c7,SystemUUID:ec259c83-36b5-f568-cf65-ad8234e187c7,BootID:0cfd40df-a240-4675-9047-6eaeea9721f4,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:e468dddcd275163a042ab297b2d8c2aca50d5e148d2d22f3b6ba119e2f31fa79 registry.k8s.io/sig-storage/csi-provisioner:v3.4.0],SizeBytes:27427836,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:9b2d9abda017c28b12f84a344f57cd73fbdb6c2bd7dd5bdd5018246ad1093ba6 registry.k8s.io/sig-storage/hostpathplugin:v1.11.0],SizeBytes:18233005,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-00731a847fa017737],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 24 23:40:39.864: INFO: Logging kubelet events for node i-0713c0b421302f422 Mar 24 23:40:39.945: INFO: Logging pods the kubelet thinks is on node i-0713c0b421302f422 Mar 24 23:40:40.020: INFO: coredns-559769c974-jmntf started at 2023-03-24 23:26:27 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container coredns ready: true, restart count 0 Mar 24 23:40:40.020: INFO: csi-mockplugin-0 started at 2023-03-24 23:39:55 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:40.020: INFO: Container csi-provisioner ready: false, restart count 0 Mar 24 23:40:40.020: INFO: Container driver-registrar ready: false, restart count 0 Mar 24 23:40:40.020: INFO: Container mock ready: false, restart count 0 Mar 24 23:40:40.020: INFO: ss-0 started at 2023-03-24 23:40:30 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container webserver ready: false, restart count 0 Mar 24 23:40:40.020: INFO: test-rs-rxnq7 started at 2023-03-24 23:40:33 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container httpd ready: true, restart count 0 Mar 24 23:40:40.020: INFO: csi-mockplugin-attacher-0 started at 2023-03-24 23:39:56 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container csi-attacher ready: false, restart count 0 Mar 24 23:40:40.020: INFO: calico-node-v5444 started at 2023-03-24 23:25:36 +0000 UTC (3+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Init container upgrade-ipam ready: true, restart count 0 Mar 24 23:40:40.020: INFO: Init container install-cni ready: true, restart count 0 Mar 24 23:40:40.020: INFO: Init container mount-bpffs ready: true, restart count 0 Mar 24 23:40:40.020: INFO: Container calico-node ready: true, restart count 0 Mar 24 23:40:40.020: INFO: pod-should-be-evicted7173a6f6-8895-42c9-bd17-f12df5b78060 started at 2023-03-24 23:40:21 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container bar ready: true, restart count 0 Mar 24 23:40:40.020: INFO: kube-proxy-i-0713c0b421302f422 started at 2023-03-24 23:25:35 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container kube-proxy ready: true, restart count 0 Mar 24 23:40:40.020: INFO: hostexec-i-0713c0b421302f422-z8vwk started at 2023-03-24 23:40:29 +0000 UTC (0+1 container statuses recorded) Mar 24 23:40:40.020: INFO: Container agnhost-container ready: true, restart count 0 Mar 24 23:40:40.020: INFO: ebs-csi-node-nc629 started at 2023-03-24 23:25:36 +0000 UTC (0+3 container statuses recorded) Mar 24 23:40:40.020: INFO: Container ebs-plugin ready: true, restart count 0 Mar 24 23:40:40.020: INFO: Container liveness-probe ready: true, restart count 0 Mar 24 23:40:40.020: INFO: Container node-driver-registrar ready: true, restart count 0 Mar 24 23:40:40.297: INFO: Latency metrics for node i-0713c0b421302f422 Mar 24 23:40:40.297: INFO: Logging node info for node i-0b03d8a3cb7d09376 Mar 24 23:40:40.364: INFO: Node Info: &Node{ObjectMeta:{i-0b03d8a3cb7d09376 8c0fad64-1a01-403f-8407-a3c2c660aca6 22479 0 2023-03-24 23:17:08 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2d kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b03d8a3cb7d09376 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2d topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2d] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b03d8a3cb7d09376"} node.alpha.kubernetes.io/ttl:0 projectcalico.org/IPv4Address:172.20.38.62/19 projectcalico.org/IPv4IPIPTunnelAddr:100.118.193.128 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-24 23:17:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-03-24 23:18:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {calico-node Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:projectcalico.org/IPv4Address":{},"f:projectcalico.org/IPv4IPIPTunnelAddr":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-24 23:18:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-03-24 23:18:32 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-24 23:40:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2d/i-0b03d8a3cb7d09376,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-24 23:18:11 +0000 UTC,LastTransitionTime:2023-03-24 23:18:11 +0000 UTC,Reason:CalicoIsUp,Message:Calico is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:17:08 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-24 23:40:15 +0000 UTC,LastTransitionTime:2023-03-24 23:18:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.62,},NodeAddress{Type:ExternalIP,Address:34.223.53.68,},NodeAddress{Type:InternalDNS,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b03d8a3cb7d09376.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-34-223-53-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec201075543ed07e71934a458e92fda2,SystemUUID:ec201075-543e-d07e-7193-4a458e92fda2,BootID:1277c323-b90c-4c0c-96e1-48ea5f1d5947,KernelVersion:5.15.0-1031-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.18,KubeletVersion:v1.28.0-alpha.0,KubeProxyVersion:v1.28.0-alpha.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[docker.io/calico/cni@sha256:e282ea2914c806b5de2976330a17cfb5e6dcef47147bceb1432ca5c75fd46f50 docker.io/calico/cni:v3.24.5],SizeBytes:87458443,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[docker.io/calico/node@sha256:5972ad2bcbdc668564d3e26960c9c513b2d7b05581c704747cf7c62ef3a405a6 docker.io/calico/node:v3.24.5],SizeBytes:81584866,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager-slim@sha256:8727adc059334e0e3329dbccce116a734da4a090da5e86c8cddccaf333caa666 registry.k8s.io/etcdadm/etcd-manager-slim:v3.0.20230201],SizeBytes:63926165,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:45219545,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:43628585,},ContainerImage{Names:[registry.k8s.io/kube-apiserver@sha256:49934a29c559e79ed19fc38b4697b18d8fe174b17bf56192588575f980c93956 registry.k8s.io/kube-apiserver:v1.28.0-alpha.0],SizeBytes:33212506,},ContainerImage{Names:[docker.io/calico/kube-controllers@sha256:2b6acd7f677f76ffe12ecf3ea7df92eb9b1bdb07336d1ac2a54c7631fb753f7e docker.io/calico/kube-controllers:v3.24.5],SizeBytes:31137169,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager@sha256:974748384634982c73527fdb3c9ec0ec476bc0d52b3dd1d611a380b7f99af1f3 registry.k8s.io/kube-controller-manager:v1.28.0-alpha.0],SizeBytes:30820280,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/kube-proxy@sha256:8a03b3147cd7be8e8235ae1cd70cf69aca9ce03ba1cfaffea320540bb5afa496 registry.k8s.io/kube-proxy:v1.28.0-alpha.0],SizeBytes:23896850,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.i