Result | FAILURE |
Tests | 57 failed / 728 succeeded |
Started | |
Elapsed | 50m7s |
Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Jan 22 02:34:01.063: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc001e58d80>: { s: "pod \"pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09\" is not Running: timed out waiting for the condition", } pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192from junit_16.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:28:58.983: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volume-expand �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] Verify if offline PVC expansion works /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174 Jan 22 02:29:00.020: INFO: Creating resource for dynamic PV Jan 22 02:29:00.021: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP�[0m: creating a StorageClass volume-expand-4780-e2e-scptxc2 �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating a pod with dynamically provisioned volume Jan 22 02:34:01.063: FAIL: While creating pods for resizing Unexpected error: <*errors.errorString | 0xc001e58d80>: { s: "pod \"pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09\" is not Running: timed out waiting for the condition", } pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:192 +0x365 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc0002ca9c0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f Jan 22 02:34:01.064: INFO: Deleting pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09" in namespace "volume-expand-4780" Jan 22 02:34:01.213: INFO: Wait up to 5m0s for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09" to be fully deleted �[1mSTEP�[0m: Deleting pod Jan 22 02:34:11.509: INFO: Deleting pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09" in namespace "volume-expand-4780" �[1mSTEP�[0m: Deleting pvc Jan 22 02:34:11.952: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comgxbr6" Jan 22 02:34:12.104: INFO: Waiting up to 3m0s for PersistentVolume pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f to get deleted Jan 22 02:34:12.252: INFO: PersistentVolume pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f found and phase=Released (147.592141ms) Jan 22 02:34:17.400: INFO: PersistentVolume pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f found and phase=Released (5.295777745s) Jan 22 02:34:22.548: INFO: PersistentVolume pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f was removed �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volume-expand-4780". �[1mSTEP�[0m: Found 19 events. Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:00 +0000 UTC - event for ebs.csi.aws.comgxbr6: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:00 +0000 UTC - event for ebs.csi.aws.comgxbr6: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:00 +0000 UTC - event for ebs.csi.aws.comgxbr6: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "volume-expand-4780/ebs.csi.aws.comgxbr6" Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:03 +0000 UTC - event for ebs.csi.aws.comgxbr6: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:04 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {default-scheduler } Scheduled: Successfully assigned volume-expand-4780/pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09 to ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:06 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f" Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:13 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f/dev" Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:13 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "pvc-261e2d65-c164-462f-9ad6-df05cf8dcd0f" volumeMapPath "/var/lib/kubelet/pods/44f9f529-40d9-4336-b352-38eadfc2d62e/volumeDevices/kubernetes.io~csi" Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:16 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f4afb24ebfbe4b667dc792076a18d379c5684ed348106a72ee65d90d5196086b" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:17 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:18 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "de148dc69e061eb64d6091ee21ee0c66621859fea7f59bd1f9d1628f0258e7dc" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:23 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "c5dcded900fa71dfa76b95f1923e0f5f295d32947b22daaa749e557bb92d41c4" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:24 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "c3fcb2caefad56f52dc00891bb87694dc32d32437b95d62fedd0736664d9e77d" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:26 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "05d63e7d97a0f9e9ab858bf9710956c827b4fdf56ff781e85ac06d33d377143b" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:27 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "8af5131e2fc846f7cc5f97cb91befdea31902c3db06150625b1745e95646c7e8" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:29 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "c016ba899ec52eb61a88e784cbc8cf368a4ec6cf69e4d56c88310d2ba1e15ae8" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:30 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "bde4b9f51c0c231dcca4c52b44c129f78b16282c205509e99492871df40b34cb" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.846: INFO: At 2023-01-22 02:29:32 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "a4306afa16f897da4d97c2c051429a3f2953773be4b9e82025199c69e50f62f0" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.847: INFO: At 2023-01-22 02:29:34 +0000 UTC - event for pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1f6714f9d785f68b765c03d3022cab6817916baf22c292049858cc84174cd32c" network for pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09": networkPlugin cni failed to set up pod "pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09_volume-expand-4780" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:22.994: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:34:22.994: INFO: Jan 22 02:34:23.144: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:23.292: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 11676 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:29:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-22 02:29:19 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-019e723d932658f17 kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-019e723d932658f17,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4,DevicePath:,},},Config:nil,},} Jan 22 02:34:23.292: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:23.442: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: netserver-0 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:23.753: INFO: test-rolling-update-controller-xhnb5 started at 2023-01-22 02:29:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container httpd ready: true, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container coredns ready: true, restart count 0 Jan 22 02:34:23.753: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-ln964 started at 2023-01-22 02:34:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: pod-80862d15-4984-4b9d-99fd-e96e46a3ac67 started at 2023-01-22 02:29:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:23.753: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:33:44 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:23.753: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:34:23.753: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:23.753: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:23.753: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:23.753: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:23.753: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:23.753: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: inline-volume-tester-rkzx7 started at 2023-01-22 02:29:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:23.753: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:30:13 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:23.753: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:34:23.753: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:23.753: INFO: test-rolling-update-deployment-8656fc4b57-c8wd7 started at 2023-01-22 02:30:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:34:23.753: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:23.753: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.319: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:24.319: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:24.467: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 13224 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4110":"csi-mock-csi-mock-volumes-4110","ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:29:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:29:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3,DevicePath:,},},Config:nil,},} Jan 22 02:34:24.467: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:24.618: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:24.787: INFO: csi-mockplugin-0 started at 2023-01-22 02:29:11 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:24.788: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:24.788: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:34:24.788: INFO: Container mock ready: false, restart count 0 Jan 22 02:34:24.788: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-58nkp started at 2023-01-22 02:33:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:24.788: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.788: INFO: test-rolling-update-with-lb-7dcdf997cd-2nzq9 started at 2023-01-22 02:30:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:34:24.788: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.788: INFO: pod-configmaps-487886f1-48d2-404e-b45d-500ec77edc3f started at 2023-01-22 02:33:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container env-test ready: false, restart count 0 Jan 22 02:34:24.788: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.788: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:24.788: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.788: INFO: dns-test-7165a663-4d55-4543-acfd-a8c273537323 started at 2023-01-22 02:27:52 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:24.788: INFO: Container jessie-querier ready: false, restart count 0 Jan 22 02:34:24.788: INFO: Container querier ready: false, restart count 0 Jan 22 02:34:24.788: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:24.788: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:24.788: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:24.788: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:24.788: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:24.788: INFO: test-rolling-update-with-lb-79bf9f6988-rqqm8 started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:24.788: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.788: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:24.788: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:24.788: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:24.789: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.789: INFO: pvc-volume-tester-plxcq started at 2023-01-22 02:29:54 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container volume-tester ready: false, restart count 0 Jan 22 02:34:24.789: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.789: INFO: pod-subpath-test-preprovisionedpv-8dnv started at 2023-01-22 02:33:59 +0000 UTC (0+2 container statuses recorded) Jan 22 02:34:24.789: INFO: Container test-container-subpath-preprovisionedpv-8dnv ready: false, restart count 0 Jan 22 02:34:24.789: INFO: Container test-container-volume-preprovisionedpv-8dnv ready: false, restart count 0 Jan 22 02:34:24.789: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.789: INFO: metadata-volume-fd30831d-8c17-454a-91a4-d45d8348548b started at 2023-01-22 02:30:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container client-container ready: false, restart count 0 Jan 22 02:34:24.789: INFO: pod-14b5f8c7-e30a-426e-8605-713988dc0cae started at 2023-01-22 02:34:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:24.789: INFO: netserver-1 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:24.789: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-2tv2s started at 2023-01-22 02:33:55 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:24.789: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:24.789: INFO: csi-mockplugin-0 started at 2023-01-22 02:27:21 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:24.789: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:24.789: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:34:24.789: INFO: Container mock ready: true, restart count 0 Jan 22 02:34:24.789: INFO: pod-projected-configmaps-12ebdd9f-aa1b-4d56-a05e-a8e067698121 started at 2023-01-22 02:29:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container projected-configmap-volume-test ready: false, restart count 0 Jan 22 02:34:24.789: INFO: csi-mockplugin-attacher-0 started at 2023-01-22 02:29:11 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:24.789: INFO: inline-volume-tester-7p6pc started at 2023-01-22 02:30:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:24.789: INFO: service-headless-xgkds started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:24.789: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:34:25.265: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:25.265: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:25.413: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 12864 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8079":"ip-172-20-47-230.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:27:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:25.414: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:25.565: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:25.737: INFO: inline-volume-tester-6nngz started at 2023-01-22 02:27:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:25.738: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:25.738: INFO: pod-5e2f43d2-6145-4e2d-8712-db9b840587e7 started at 2023-01-22 02:30:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:25.738: INFO: service-headless-p8n6b started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:34:25.738: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:34:25.738: INFO: test-rolling-update-with-lb-79bf9f6988-44b8c started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:27:08 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:25.738: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-dsw6d started at 2023-01-22 02:30:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:25.738: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:25.738: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:25.738: INFO: inline-volume-tester2-zmfh6 started at 2023-01-22 02:29:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:25.738: INFO: netserver-2 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:25.738: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:25.738: INFO: Container coredns ready: true, restart count 0 Jan 22 02:34:26.328: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:26.328: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:26.477: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 13046 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:28:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:26.477: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:26.629: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:26.785: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: test-rolling-update-with-lb-79bf9f6988-4fp4m started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: pod-configmaps-220ddd82-1429-45eb-b1eb-7f5627bdf54e started at 2023-01-22 02:29:58 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:26.786: INFO: busybox-5e27475f-0f54-40db-a2ce-a1c9dee0380c started at 2023-01-22 02:32:42 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container busybox ready: true, restart count 0 Jan 22 02:34:26.786: INFO: netserver-3 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:26.786: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:26.786: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:26.786: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:26.786: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:26.786: INFO: service-headless-w4f7z started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container service-headless ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:26.786: INFO: pod-906bd654-ba45-4cea-8ad3-0065821bb928 started at 2023-01-22 02:33:49 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:26.786: INFO: Container test-container ready: false, restart count 0 Jan 22 02:34:27.263: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:27.263: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:27.411: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 12105 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:27.412: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:27.562: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:27.718: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.718: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:34:27.718: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.718: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:34:27.718: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.718: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:34:27.718: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:27.718: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:27.718: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:27.718: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.718: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:34:27.718: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.719: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:34:27.719: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:34:27.719: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:34:27.719: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.719: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:34:27.719: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:27.719: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:34:27.719: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:27.719: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:27.719: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:34:27.719: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:27.719: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:28.181: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:28.181: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volume-expand-4780" for this suite.
Find pod-0043eae6-6fb0-43a7-a6a6-5c1ef4e8af09 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\snew\spod\sfsgroup\sapplied\sto\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:214 Jan 22 02:46:02.849: Unexpected error: <*errors.errorString | 0xc0006f7300>: { s: "pod \"pod-efed6c88-76fd-4889-8a42-f736d44b87a6\" is not Running: timed out waiting for the condition", } pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:262from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:38:36.206: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:214 Jan 22 02:38:37.249: INFO: Creating resource for dynamic PV Jan 22 02:38:37.249: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-3457-e2e-scjb5c4 �[1mSTEP�[0m: creating a claim Jan 22 02:38:37.401: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-3457 with fsgroup 1000 Jan 22 02:40:52.303: INFO: Pod fsgroupchangepolicy-3457/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 started successfully �[1mSTEP�[0m: Creating a sub-directory and file, and verifying their ownership is 1000 Jan 22 02:40:52.303: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/file1] Namespace:fsgroupchangepolicy-3457 PodName:pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Jan 22 02:40:52.303: INFO: >>> kubeConfig: /root/.kube/config Jan 22 02:40:52.304: INFO: ExecWithOptions: Clientset creation Jan 22 02:40:52.304: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-deb10-k23-docker.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-3457/pods/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3/exec?command=%2Fbin%2Fsh&command=-c&command=touch+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true %!s(MISSING)) Jan 22 02:40:53.483: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/file1] Namespace:fsgroupchangepolicy-3457 PodName:pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Jan 22 02:40:53.483: INFO: >>> kubeConfig: /root/.kube/config Jan 22 02:40:53.484: INFO: ExecWithOptions: Clientset creation Jan 22 02:40:53.484: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-deb10-k23-docker.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-3457/pods/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true %!s(MISSING)) Jan 22 02:40:54.508: INFO: pod fsgroupchangepolicy-3457/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 exec for cmd ls -l /mnt/volume1/file1, stdout: -rw-r--r-- 1 root 1000 0 Jan 22 02:40 /mnt/volume1/file1, stderr: Jan 22 02:40:54.509: INFO: stdout split: [-rw-r--r-- 1 root 1000 0 Jan 22 02:40 /mnt/volume1/file1], expected gid: 1000 Jan 22 02:40:54.509: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir /mnt/volume1/subdir] Namespace:fsgroupchangepolicy-3457 PodName:pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Jan 22 02:40:54.509: INFO: >>> kubeConfig: /root/.kube/config Jan 22 02:40:54.509: INFO: ExecWithOptions: Clientset creation Jan 22 02:40:54.509: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-deb10-k23-docker.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-3457/pods/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3/exec?command=%2Fbin%2Fsh&command=-c&command=mkdir+%2Fmnt%2Fvolume1%2Fsubdir&container=write-pod&container=write-pod&stderr=true&stdout=true %!s(MISSING)) Jan 22 02:40:55.552: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-3457 PodName:pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Jan 22 02:40:55.552: INFO: >>> kubeConfig: /root/.kube/config Jan 22 02:40:55.552: INFO: ExecWithOptions: Clientset creation Jan 22 02:40:55.553: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-deb10-k23-docker.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-3457/pods/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3/exec?command=%2Fbin%2Fsh&command=-c&command=touch+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true %!s(MISSING)) Jan 22 02:40:56.697: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-3457 PodName:pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Jan 22 02:40:56.697: INFO: >>> kubeConfig: /root/.kube/config Jan 22 02:40:56.698: INFO: ExecWithOptions: Clientset creation Jan 22 02:40:56.698: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-deb10-k23-docker.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-3457/pods/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true %!s(MISSING)) Jan 22 02:40:57.790: INFO: pod fsgroupchangepolicy-3457/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r-- 1 root 1000 0 Jan 22 02:40 /mnt/volume1/subdir/file2, stderr: Jan 22 02:40:57.790: INFO: stdout split: [-rw-r--r-- 1 root 1000 0 Jan 22 02:40 /mnt/volume1/subdir/file2], expected gid: 1000 �[1mSTEP�[0m: Deleting Pod fsgroupchangepolicy-3457/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 Jan 22 02:40:57.790: INFO: Deleting pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3" in namespace "fsgroupchangepolicy-3457" Jan 22 02:40:57.943: INFO: Wait up to 5m0s for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3" to be fully deleted �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-3457 with fsgroup 2000 Jan 22 02:46:02.849: FAIL: Unexpected error: <*errors.errorString | 0xc0006f7300>: { s: "pod \"pod-efed6c88-76fd-4889-8a42-f736d44b87a6\" is not Running: timed out waiting for the condition", } pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001bd8f20, 0xc001731068, 0x0, {0xc000c678f8, 0x4}, {0xc000c678fc, 0x4}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:262 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:251 +0x556 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x0?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc000425ba0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f �[1mSTEP�[0m: Deleting pvc Jan 22 02:46:03.147: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comhwjcz" Jan 22 02:46:03.300: INFO: Waiting up to 3m0s for PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 to get deleted Jan 22 02:46:03.449: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (149.068421ms) Jan 22 02:46:08.599: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (5.298779283s) Jan 22 02:46:13.749: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (10.448632267s) Jan 22 02:46:18.898: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (15.597951841s) Jan 22 02:46:24.047: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (20.747428983s) Jan 22 02:46:29.196: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (25.896435427s) Jan 22 02:46:34.347: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (31.046910596s) Jan 22 02:46:39.497: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (36.196652001s) Jan 22 02:46:44.649: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (41.349299s) Jan 22 02:46:49.800: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (46.499741185s) Jan 22 02:46:54.951: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (51.650541613s) Jan 22 02:47:00.101: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (56.801047536s) Jan 22 02:47:05.254: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m1.95422294s) Jan 22 02:47:10.404: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m7.104349838s) Jan 22 02:47:15.556: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m12.255583737s) Jan 22 02:47:20.707: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m17.407089172s) Jan 22 02:47:25.857: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m22.557460642s) Jan 22 02:47:31.013: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m27.71273495s) Jan 22 02:47:36.163: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m32.863013113s) Jan 22 02:47:41.314: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m38.014254622s) Jan 22 02:47:46.466: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m43.166322176s) Jan 22 02:47:51.618: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m48.318187175s) Jan 22 02:47:56.767: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m53.467432799s) Jan 22 02:48:01.923: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (1m58.622784784s) Jan 22 02:48:07.076: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m3.775533239s) Jan 22 02:48:12.227: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m8.926824648s) Jan 22 02:48:17.377: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m14.076685171s) Jan 22 02:48:22.528: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m19.228131253s) Jan 22 02:48:27.677: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m24.377190673s) Jan 22 02:48:32.827: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m29.526823227s) Jan 22 02:48:37.977: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m34.677087889s) Jan 22 02:48:43.127: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m39.826612774s) Jan 22 02:48:48.276: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m44.975934265s) Jan 22 02:48:53.426: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m50.126447067s) Jan 22 02:48:58.575: INFO: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 found and phase=Bound (2m55.275401091s) �[1mSTEP�[0m: Deleting sc Jan 22 02:49:03.729: FAIL: while cleanup resource Unexpected error: <errors.aggregate | len:1, cap:1>: [ [ { msg: "persistent Volume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 not deleted by dynamic provisioner: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 still exists within 3m0s", err: { s: "PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 still exists within 3m0s", }, }, ], ] persistent Volume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 not deleted by dynamic provisioner: PersistentVolume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 still exists within 3m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0x24c panic({0x6bb1ac0, 0xc003432f40}) /usr/local/go/src/runtime/panic.go:884 +0x212 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x623d460, 0x78c75a0}) /usr/local/go/src/runtime/panic.go:884 +0x212 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0005328c0, 0x13b}, {0xc0021f2d30?, 0x7047513?, 0xc0021f2d50?}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc000532780, 0x126}, {0xc00355ac40?, 0xc000532780?, 0xc003e8e1a0?}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc0021f2e98, {0x78f18f8, 0xa516880}, 0x0, {0x0, 0x0, 0x0}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc0021f2e98, {0x78f18f8, 0xa516880}, {0x0, 0x0, 0x0}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7938928?, {0x78cb4e0?, 0xc0006f7300?}, {0x0?, 0x2?, 0x0?}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/expect.go:46 +0x9d k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/expect.go:40 k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001bd8f20, 0xc001731068, 0x0, {0xc000c678f8, 0x4}, {0xc000c678fc, 0x4}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:262 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:251 +0x556 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x0?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc000425ba0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-3457". �[1mSTEP�[0m: Found 29 events. Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:37 +0000 UTC - event for ebs.csi.aws.comhwjcz: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:37 +0000 UTC - event for ebs.csi.aws.comhwjcz: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:37 +0000 UTC - event for ebs.csi.aws.comhwjcz: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-3457/ebs.csi.aws.comhwjcz" Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:41 +0000 UTC - event for ebs.csi.aws.comhwjcz: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7 Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:41 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-3457/pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 to ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:44 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-7cc9581a-f9d2-420e-af65-7cedd72134e7" Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:46 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ddc566af309d531e1c32405afbbdfefe136db0fed0e05793851b2a07d8d62c24" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:48 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:49 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "74e7c1af08512fdc8fc4a559606f8de7e8b5f49166b281645581508b57d12998" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:50 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "267310f8f8fe460595d29a434df6ed681bb479f08c3ae43b146d774bb1c6b687" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:51 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "d3cf4ed71bb17e9581b1fb47233d0c2183ef4d3cba7696153d5f3be2e67aac35" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:52 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "fe341661df20c3ce4fe1ffc9486258d3b0835f13d46e21e3bea345af6c1846e7" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:54 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ae93937c0d1951ec201e04db79a0928ab14f013fd9c43b8db2cc2ecb115da001" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:55 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "188f9b650e852e99d14309afe4e597e0b34f80c0a2a02e03c3e8f2a48fa754c1" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:57 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "d3c9d121df9db622fa867edee292ab5f2f773c9da1bb480873ba7c5366c38576" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:38:59 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "eae4f1c10ee75a303d0b083ba60afce8a141e863d12dd528d6f924f7c6664ed3" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:39:01 +0000 UTC - event for pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "5525505118bdee6d05c867edba2f38a9f028cf44172f54e6c54d19925094b530" network for pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3": networkPlugin cni failed to set up pod "pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:02 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-3457/pod-efed6c88-76fd-4889-8a42-f736d44b87a6 to ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:09 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "88d014be96ec3609412d03fe15cbb920f59c178edded52013ace0d152f1405d5" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:11 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:13 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "6320c4c14ed2aa310af42d530b4b25e2807a1fdb352cccf5885851949dbf4036" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:17 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "b0d5b5e89928aa693937385b2b75650c245ee9d13a340fd6744b7b6ac39ee6cc" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:20 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ac12efea56d4ded59ac4a802415e26484c4cc920f28e73b15e778ad447b714fb" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:24 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "6663001dadec34a53058faaf3ad4f6f04c3711d0e31918691180139909907c64" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:29 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "bc2c65b858467ebbc003de274c0f09812d2a3ca8f45c8b03fadc8a2ac4296cb0" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:33 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "57fedb6e6e9380138011e4ff95b7fd5dc877b26ce8c0d68ed195c861a1cf19a7" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:37 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "fcb87130e54dd11e357d94512c6462b2251599db07599a96df063b0fa8affe18" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:40 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "7acc02530795ddf9d6bff7dbb5da8366f069a21f1904e1239ce8f25d2e42d0ca" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:03.882: INFO: At 2023-01-22 02:41:44 +0000 UTC - event for pod-efed6c88-76fd-4889-8a42-f736d44b87a6: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "bb2072423b342aeb02084ca33b3053de08928e7896dac4cdea5614b3d755da38" network for pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6": networkPlugin cni failed to set up pod "pod-efed6c88-76fd-4889-8a42-f736d44b87a6_fsgroupchangepolicy-3457" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:49:04.031: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:49:04.031: INFO: pod-efed6c88-76fd-4889-8a42-f736d44b87a6 ip-172-20-54-133.ap-northeast-1.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:41:02 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:46:37 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:46:37 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:41:02 +0000 UTC }] Jan 22 02:49:04.031: INFO: Jan 22 02:49:04.358: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:49:04.507: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 25301 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volumemode-7302":"ip-172-20-33-190.ap-northeast-1.compute.internal","csi-mock-csi-mock-volumes-7789":"csi-mock-csi-mock-volumes-7789","ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-01-22 02:48:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:46 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:46 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:46 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:48:46 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:99c0d6f1ad24a1aa1905d9c6534d193f268f7b23f9add2ae6bb41f31094bdd5c k8s.gcr.io/e2e-test-images/nautilus:1.5],SizeBytes:124758741,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:49:04.508: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:49:04.663: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:49:04.832: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:48:10 +0000 UTC (0+7 container statuses recorded) Jan 22 02:49:04.832: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:49:04.832: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:49:04.832: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:49:04.832: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:49:04.832: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:49:04.832: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:04.832: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:49:04.832: INFO: ss2-1 started at 2023-01-22 02:48:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container webserver ready: true, restart count 0 Jan 22 02:49:04.832: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container coredns ready: true, restart count 0 Jan 22 02:49:04.832: INFO: netserver-0 started at 2023-01-22 02:48:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container webserver ready: true, restart count 0 Jan 22 02:49:04.832: INFO: deployment-shared-unset-c757c87b9-tm9vr started at 2023-01-22 02:47:43 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container nginx ready: false, restart count 0 Jan 22 02:49:04.832: INFO: deployment-shared-unset-c757c87b9-wrx8r started at 2023-01-22 02:47:43 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container nginx ready: false, restart count 0 Jan 22 02:49:04.832: INFO: netserver-0 started at 2023-01-22 02:49:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:04.832: INFO: pvc-volume-tester-5stkc started at 2023-01-22 02:48:43 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container volume-tester ready: true, restart count 0 Jan 22 02:49:04.832: INFO: netserver-0 started at 2023-01-22 02:48:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.832: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:04.832: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:49:04.833: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:49:04.833: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:49:04.833: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:04.833: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:49:04.833: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:04.833: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:49:04.833: INFO: adopt-release-tqhr6 started at 2023-01-22 02:47:42 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:04.833: INFO: Container c ready: false, restart count 0 Jan 22 02:49:04.833: INFO: csi-mockplugin-0 started at 2023-01-22 02:48:10 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:04.833: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:49:04.833: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:49:04.833: INFO: Container mock ready: true, restart count 0 Jan 22 02:49:05.318: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:49:05.318: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:49:05.469: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 24997 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-39-135.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1842":"csi-mock-csi-mock-volumes-1842","ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:48:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:09 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:09 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:09 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:48:09 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0507e755b447408f6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0507e755b447408f6,DevicePath:,},},Config:nil,},} Jan 22 02:49:05.469: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:49:05.620: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:49:05.775: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:05.775: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:49:05.775: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:05.775: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:49:05.775: INFO: test-rs-t66c6 started at <nil> (0+0 container statuses recorded) Jan 22 02:49:05.776: INFO: csi-mockplugin-resizer-0 started at 2023-01-22 02:42:35 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:49:05.776: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:49:05.776: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:49:05.776: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-txvcm started at 2023-01-22 02:48:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:49:05.776: INFO: pod-0268e4a7-6915-4a5f-9a13-203859adb942 started at 2023-01-22 02:48:27 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container write-pod ready: true, restart count 0 Jan 22 02:49:05.776: INFO: netserver-1 started at 2023-01-22 02:48:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:05.776: INFO: test-container-pod started at 2023-01-22 02:48:36 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:05.776: INFO: inline-volume-tester-2jdp7 started at 2023-01-22 02:48:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:49:05.776: INFO: pod-subpath-test-inlinevolume-m49s started at 2023-01-22 02:48:50 +0000 UTC (2+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Init container init-volume-inlinevolume-m49s ready: false, restart count 0 Jan 22 02:49:05.776: INFO: Init container test-init-volume-inlinevolume-m49s ready: false, restart count 0 Jan 22 02:49:05.776: INFO: Container test-container-subpath-inlinevolume-m49s ready: false, restart count 0 Jan 22 02:49:05.776: INFO: ss2-0 started at <nil> (0+0 container statuses recorded) Jan 22 02:49:05.776: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-hbxbv started at 2023-01-22 02:48:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:49:05.776: INFO: netserver-1 started at 2023-01-22 02:48:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container webserver ready: true, restart count 0 Jan 22 02:49:05.776: INFO: netserver-1 started at <nil> (0+0 container statuses recorded) Jan 22 02:49:05.776: INFO: webserver-pod started at 2023-01-22 02:46:52 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:49:05.776: INFO: pvc-volume-tester-sdddx started at 2023-01-22 02:48:20 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:05.776: INFO: Container volume-tester ready: true, restart count 0 Jan 22 02:49:05.776: INFO: csi-mockplugin-0 started at 2023-01-22 02:42:35 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:05.776: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:49:05.776: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:49:05.776: INFO: Container mock ready: true, restart count 0 Jan 22 02:49:06.278: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:49:06.278: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:49:06.431: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 23903 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:31 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:31 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:31 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:47:31 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:99c0d6f1ad24a1aa1905d9c6534d193f268f7b23f9add2ae6bb41f31094bdd5c k8s.gcr.io/e2e-test-images/nautilus:1.5],SizeBytes:124758741,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:49:06.432: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:49:06.583: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:49:06.748: INFO: pfpod started at 2023-01-22 02:48:53 +0000 UTC (0+2 container statuses recorded) Jan 22 02:49:06.748: INFO: Container portforwardtester ready: false, restart count 0 Jan 22 02:49:06.748: INFO: Container readiness ready: true, restart count 0 Jan 22 02:49:06.748: INFO: rs-qcs8g started at 2023-01-22 02:47:00 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container donothing ready: false, restart count 0 Jan 22 02:49:06.748: INFO: netserver-2 started at 2023-01-22 02:48:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container webserver ready: true, restart count 0 Jan 22 02:49:06.748: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-bkc8f started at 2023-01-22 02:48:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:49:06.748: INFO: netserver-2 started at 2023-01-22 02:48:48 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:06.748: INFO: pod-configmaps-4ea8026c-f3ca-47db-a97e-92d0ed50f701 started at 2023-01-22 02:48:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:49:06.748: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:06.748: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:49:06.748: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:06.748: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:49:06.748: INFO: update-demo-nautilus-l9qhw started at 2023-01-22 02:48:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container update-demo ready: true, restart count 0 Jan 22 02:49:06.748: INFO: startup-f716a3c2-207a-4ab2-b80d-ee7ae82cbe98 started at 2023-01-22 02:47:56 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container busybox ready: false, restart count 0 Jan 22 02:49:06.748: INFO: test-rs-jrqtw started at 2023-01-22 02:49:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container httpd ready: false, restart count 0 Jan 22 02:49:06.748: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:49:06.748: INFO: deployment-shared-unset-c757c87b9-wkrx8 started at 2023-01-22 02:47:43 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container nginx ready: false, restart count 0 Jan 22 02:49:06.748: INFO: netserver-2 started at 2023-01-22 02:49:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:06.748: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:49:06.748: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:49:06.748: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container coredns ready: true, restart count 0 Jan 22 02:49:06.748: INFO: pod-secrets-ac973213-503f-4fa2-96f6-207bbbe4f21d started at 2023-01-22 02:48:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:06.748: INFO: Container secret-env-test ready: false, restart count 0 Jan 22 02:49:07.239: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:49:07.239: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:49:07.389: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 24614 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:38:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:38:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:41 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:41 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:48:41 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:48:41 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nautilus@sha256:99c0d6f1ad24a1aa1905d9c6534d193f268f7b23f9add2ae6bb41f31094bdd5c k8s.gcr.io/e2e-test-images/nautilus:1.5],SizeBytes:124758741,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8,DevicePath:,},},Config:nil,},} Jan 22 02:49:07.389: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:49:07.540: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:49:07.696: INFO: netserver-3 started at 2023-01-22 02:48:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.696: INFO: Container webserver ready: true, restart count 0 Jan 22 02:49:07.696: INFO: test-webserver-101d4dee-6538-4fa4-baf9-dc34a3426624 started at 2023-01-22 02:48:20 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.696: INFO: Container test-webserver ready: false, restart count 0 Jan 22 02:49:07.697: INFO: pod-efed6c88-76fd-4889-8a42-f736d44b87a6 started at 2023-01-22 02:41:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container write-pod ready: true, restart count 0 Jan 22 02:49:07.697: INFO: hostexec-ip-172-20-54-133.ap-northeast-1.compute.internal-kplj6 started at 2023-01-22 02:48:28 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:49:07.697: INFO: ss2-2 started at 2023-01-22 02:48:36 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container webserver ready: true, restart count 0 Jan 22 02:49:07.697: INFO: pause-pod-1 started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:49:07.697: INFO: privileged-pod started at 2023-01-22 02:48:57 +0000 UTC (0+2 container statuses recorded) Jan 22 02:49:07.697: INFO: Container not-privileged-container ready: true, restart count 0 Jan 22 02:49:07.697: INFO: Container privileged-container ready: true, restart count 0 Jan 22 02:49:07.697: INFO: netserver-3 started at 2023-01-22 02:48:48 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:07.697: INFO: sysctl-99a25c41-8104-4ed1-a75a-a432c5cfdc70 started at 2023-01-22 02:49:03 +0000 UTC (0+0 container statuses recorded) Jan 22 02:49:07.697: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:07.697: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:49:07.697: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:07.697: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:49:07.697: INFO: annotationupdate2521e65e-3e07-44d0-9579-23e032ae599d started at 2023-01-22 02:48:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container client-container ready: true, restart count 0 Jan 22 02:49:07.697: INFO: downward-api-9fa5d83d-be9b-4fac-849d-99474f7e0655 started at 2023-01-22 02:49:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container dapi-container ready: false, restart count 0 Jan 22 02:49:07.697: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:49:07.697: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:49:07.697: INFO: pod-subpath-test-preprovisionedpv-dtcs started at 2023-01-22 02:48:58 +0000 UTC (2+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Init container init-volume-preprovisionedpv-dtcs ready: true, restart count 0 Jan 22 02:49:07.697: INFO: Init container test-init-subpath-preprovisionedpv-dtcs ready: true, restart count 0 Jan 22 02:49:07.697: INFO: Container test-container-subpath-preprovisionedpv-dtcs ready: false, restart count 0 Jan 22 02:49:07.697: INFO: update-demo-nautilus-wtxzr started at 2023-01-22 02:48:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container update-demo ready: true, restart count 0 Jan 22 02:49:07.697: INFO: test-rs-qxc5j started at 2023-01-22 02:48:57 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container httpd ready: true, restart count 0 Jan 22 02:49:07.697: INFO: netserver-3 started at 2023-01-22 02:49:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:07.697: INFO: Container webserver ready: false, restart count 0 Jan 22 02:49:08.196: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:49:08.196: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:49:08.346: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 19078 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:49:08.346: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:49:08.497: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:49:08.654: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:49:08.655: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:49:08.655: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:49:08.655: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:49:08.655: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:49:08.655: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:49:08.655: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:49:08.655: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:49:08.655: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:49:08.655: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:49:08.655: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:49:08.655: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:49:08.655: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:49:08.655: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:49:08.655: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:49:08.655: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:49:09.128: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:49:09.128: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-3457" for this suite.
Find pod-efed6c88-76fd-4889-8a42-f736d44b87a6 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(OnRootMismatch\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\svia\schgrp\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sskips\sownership\schanges\sto\sthe\svolume\scontents$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:214 Jan 22 02:42:03.255: Unexpected error: <*errors.errorString | 0xc0041b91c0>: { s: "pod \"pod-c1563870-772d-4914-a1a9-6ae1736b51eb\" is not Running: timed out waiting for the condition", } pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:262from junit_16.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:37:01.168: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename fsgroupchangepolicy �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup skips ownership changes to the volume contents /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:214 Jan 22 02:37:02.205: INFO: Creating resource for dynamic PV Jan 22 02:37:02.205: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass fsgroupchangepolicy-9092-e2e-scw9prh �[1mSTEP�[0m: creating a claim Jan 22 02:37:02.355: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating Pod in namespace fsgroupchangepolicy-9092 with fsgroup 1000 Jan 22 02:42:03.255: FAIL: Unexpected error: <*errors.errorString | 0xc0041b91c0>: { s: "pod \"pod-c1563870-772d-4914-a1a9-6ae1736b51eb\" is not Running: timed out waiting for the condition", } pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001ce3080, 0xc0043a3068, 0x1, {0x0, 0x0}, {0x0, 0x0}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:262 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:233 +0x298 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc0002ca9c0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f �[1mSTEP�[0m: Deleting pvc Jan 22 02:42:03.551: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comgx7lr" Jan 22 02:42:03.701: INFO: Waiting up to 3m0s for PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f to get deleted Jan 22 02:42:03.848: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (147.65742ms) Jan 22 02:42:08.997: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (5.296196068s) Jan 22 02:42:14.146: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (10.445046488s) Jan 22 02:42:19.295: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (15.593876955s) Jan 22 02:42:24.443: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (20.741951656s) Jan 22 02:42:29.591: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (25.890365285s) Jan 22 02:42:34.739: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (31.038657428s) Jan 22 02:42:39.889: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (36.187902893s) Jan 22 02:42:45.037: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (41.33609291s) Jan 22 02:42:50.185: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (46.484314133s) Jan 22 02:42:55.334: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (51.633306243s) Jan 22 02:43:00.483: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (56.782718535s) Jan 22 02:43:05.632: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m1.931029672s) Jan 22 02:43:10.782: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m7.081352453s) Jan 22 02:43:15.932: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m12.231748247s) Jan 22 02:43:21.083: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m17.38211734s) Jan 22 02:43:26.233: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m22.532470668s) Jan 22 02:43:31.381: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m27.68045969s) Jan 22 02:43:36.529: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m32.828718967s) Jan 22 02:43:41.680: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m37.979175267s) Jan 22 02:43:46.832: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m43.131058656s) Jan 22 02:43:51.984: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m48.282869898s) Jan 22 02:43:57.132: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m53.430931025s) Jan 22 02:44:02.284: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (1m58.583113652s) Jan 22 02:44:07.432: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m3.731780834s) Jan 22 02:44:12.582: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m8.881732247s) Jan 22 02:44:17.734: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m14.033347573s) Jan 22 02:44:22.884: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m19.183352121s) Jan 22 02:44:28.034: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m24.333697797s) Jan 22 02:44:33.184: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m29.482980858s) Jan 22 02:44:38.332: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m34.631276958s) Jan 22 02:44:43.480: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m39.779777947s) Jan 22 02:44:48.629: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m44.928296925s) Jan 22 02:44:53.777: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m50.076347354s) Jan 22 02:44:58.925: INFO: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f found and phase=Bound (2m55.224673023s) �[1mSTEP�[0m: Deleting sc Jan 22 02:45:04.077: FAIL: while cleanup resource Unexpected error: <errors.aggregate | len:1, cap:1>: [ [ { msg: "persistent Volume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f not deleted by dynamic provisioner: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f still exists within 3m0s", err: { s: "PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f still exists within 3m0s", }, }, ], ] persistent Volume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f not deleted by dynamic provisioner: PersistentVolume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f still exists within 3m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0x24c panic({0x6bb1ac0, 0xc0049c2fc0}) /usr/local/go/src/runtime/panic.go:884 +0x212 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x623d460, 0x78c75a0}) /usr/local/go/src/runtime/panic.go:884 +0x212 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc003f60140, 0x13b}, {0xc001756d30?, 0x7047513?, 0xc001756d50?}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc003f60000, 0x126}, {0xc002100170?, 0xc003f60000?, 0xc00253a0d0?}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc001756e98, {0x78f18f8, 0xa516880}, 0x0, {0x0, 0x0, 0x0}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc001756e98, {0x78f18f8, 0xa516880}, {0x0, 0x0, 0x0}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7938928?, {0x78cb4e0?, 0xc0041b91c0?}, {0x0?, 0x2?, 0x7fecf6b41a68?}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/expect.go:46 +0x9d k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/expect.go:40 k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001ce3080, 0xc0043a3068, 0x1, {0x0, 0x0}, {0x0, 0x0}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:262 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:233 +0x298 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc0002ca9c0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "fsgroupchangepolicy-9092". �[1mSTEP�[0m: Found 17 events. Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:02 +0000 UTC - event for ebs.csi.aws.comgx7lr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:02 +0000 UTC - event for ebs.csi.aws.comgx7lr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:02 +0000 UTC - event for ebs.csi.aws.comgx7lr: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-9092/ebs.csi.aws.comgx7lr" Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:06 +0000 UTC - event for ebs.csi.aws.comgx7lr: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:06 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-9092/pod-c1563870-772d-4914-a1a9-6ae1736b51eb to ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:08 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-ac122f47-502a-4f9d-9ace-5f0c3c97287f" Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:16 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "145d6ff9a5d7bf701eb8bae53d5d3ad2302c3c67f0b80952ce1dbe1c6e3991c9" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:16 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:18 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "e741c1ca4fc7850a3b7c30219fe64fb998b54da6490138b6abb3613d07736938" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:20 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "7972d0d647103ce4a726a9bcabf483bd27b60afc68502accca5a9d4696790c41" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:23 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1ac007df4a48455b054558a0e40b1797afca9d73d8fbcc7cea6eef4e3bf9c693" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:25 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ae7e1def12221d013b622a2a679b0ecf5c91456ae62a092663e75cc655b4d0d0" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:26 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "b792804e56a23dd478c5f36844ca589c1e89f103d441c8cfcf17b78f5171aaf6" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:28 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "eee3cc6c5f15834fe60fd71da613bb919b632b5c533d28e04edcbcb920ab3928" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:31 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "735d7273540f7b95048a43501458c810c253870470719bb19dbac7d75eda09e7" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:34 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "15f2bc3f90109a2663bdfca7b0a08a502f20acb4fc0dfea3c336f4119f38b39a" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.227: INFO: At 2023-01-22 02:37:36 +0000 UTC - event for pod-c1563870-772d-4914-a1a9-6ae1736b51eb: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "030b5be1195194cf017057bc6afe8eef8ff8bb3e89d0c2a6705779cc1c28c8d9" network for pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb": networkPlugin cni failed to set up pod "pod-c1563870-772d-4914-a1a9-6ae1736b51eb_fsgroupchangepolicy-9092" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:04.375: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:45:04.375: INFO: pod-c1563870-772d-4914-a1a9-6ae1736b51eb ip-172-20-39-135.ap-northeast-1.compute.internal Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:37:06 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:37:06 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:37:06 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:37:06 +0000 UTC }] Jan 22 02:45:04.375: INFO: Jan 22 02:45:04.527: INFO: Unable to fetch fsgroupchangepolicy-9092/pod-c1563870-772d-4914-a1a9-6ae1736b51eb/write-pod logs: the server rejected our request for an unknown reason (get pods pod-c1563870-772d-4914-a1a9-6ae1736b51eb) Jan 22 02:45:04.677: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:04.825: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 18484 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:44:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:45:04.826: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:04.976: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:05.133: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.133: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:05.134: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:05.134: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:05.134: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:05.134: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:05.134: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:05.134: INFO: pod-5f304ab5-0d6e-44a5-aef0-099d371557a8 started at 2023-01-22 02:40:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: netserver-0 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-pwlp4 started at 2023-01-22 02:41:33 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:05.134: INFO: rs-tcq62 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container donothing ready: false, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-kh792 started at 2023-01-22 02:40:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:05.134: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Container coredns ready: true, restart count 0 Jan 22 02:45:05.134: INFO: pod-subpath-test-preprovisionedpv-89x4 started at 2023-01-22 02:41:44 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:05.134: INFO: Init container init-volume-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:05.134: INFO: Init container test-init-volume-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:05.134: INFO: Container test-container-subpath-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:05.611: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:05.611: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:05.759: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 17813 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:29:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:29:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-02f9a67e9fa46c78f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02f9a67e9fa46c78f,DevicePath:,},},Config:nil,},} Jan 22 02:45:05.760: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:05.910: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: hostpathsymlink-client started at 2023-01-22 02:40:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container hostpathsymlink-client ready: false, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: indexed-job-0-qzndn started at 2023-01-22 02:41:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container c ready: false, restart count 0 Jan 22 02:45:06.069: INFO: csi-mockplugin-0 started at 2023-01-22 02:42:35 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:06.069: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container mock ready: false, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:06.069: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:06.069: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:06.069: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: rs-sp685 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container donothing ready: true, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: csi-mockplugin-resizer-0 started at 2023-01-22 02:42:35 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:45:06.069: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:06.069: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: indexed-job-1-dcvvg started at 2023-01-22 02:41:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container c ready: false, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:06.069: INFO: indexed-job-2-2cddx started at 2023-01-22 02:42:12 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container c ready: false, restart count 0 Jan 22 02:45:06.069: INFO: netserver-1 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:06.069: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:10 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:06.069: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:45:06.069: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:45:06.069: INFO: pod-c1563870-772d-4914-a1a9-6ae1736b51eb started at 2023-01-22 02:37:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:06.069: INFO: inline-volume-tester-9rmrs started at 2023-01-22 02:41:10 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:06.069: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:45:06.560: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:06.561: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:06.709: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 17716 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:41:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:41:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0371bc02787c59471],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0371bc02787c59471,DevicePath:,},},Config:nil,},} Jan 22 02:45:06.709: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:06.860: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:07.032: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-r4jz4 started at 2023-01-22 02:41:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:07.032: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:07.032: INFO: test-rs-58mcf started at 2023-01-22 02:41:33 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container httpd ready: false, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: echo-sourceip started at 2023-01-22 02:41:28 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:45:07.032: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container coredns ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: netserver-2 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: pod-subpath-test-preprovisionedpv-5xtv started at 2023-01-22 02:41:59 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Init container init-volume-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:07.032: INFO: Init container test-init-volume-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:07.032: INFO: Container test-container-subpath-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:07.032: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:07.032: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:07.032: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:07.032: INFO: pod-subpath-test-inlinevolume-zhsl started at 2023-01-22 02:41:49 +0000 UTC (1+2 container statuses recorded) Jan 22 02:45:07.032: INFO: Init container test-init-subpath-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:07.032: INFO: Container test-container-subpath-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:07.032: INFO: Container test-container-volume-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.032: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:45:07.032: INFO: pod-72c4e40c-da11-4c01-bd78-334fabe5b47e started at 2023-01-22 02:41:46 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.032: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:07.594: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:07.594: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:07.742: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 17489 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5291":"ip-172-20-54-133.ap-northeast-1.compute.internal","csi-hostpath-provisioning-6245":"ip-172-20-54-133.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:38:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:38:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5291^35f2d965-99fe-11ed-a40a-3e2b90db047f kubernetes.io/csi/csi-hostpath-provisioning-6245^47c4b974-99fe-11ed-8f9c-42b9bcafcd1e kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5291^35f2d965-99fe-11ed-a40a-3e2b90db047f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6245^47c4b974-99fe-11ed-8f9c-42b9bcafcd1e,DevicePath:,},},Config:nil,},} Jan 22 02:45:07.742: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:07.892: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:08.055: INFO: pod-subpath-test-inlinevolume-lm48 started at 2023-01-22 02:41:00 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Init container init-volume-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Init container test-init-volume-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-subpath-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: pod-subpath-test-inlinevolume-775p started at 2023-01-22 02:41:24 +0000 UTC (1+2 container statuses recorded) Jan 22 02:45:08.055: INFO: Init container test-init-subpath-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-subpath-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-volume-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: hostpath-injector started at 2023-01-22 02:41:32 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container hostpath-injector ready: false, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:40:50 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:08.055: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:08.055: INFO: pod-subpath-test-dynamicpv-5kk9 started at 2023-01-22 02:41:04 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Init container init-volume-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Init container test-init-subpath-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-subpath-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: image-pull-test09b56348-f546-4734-be6b-566fb99d2d3e started at 2023-01-22 02:41:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container image-pull-test ready: false, restart count 0 Jan 22 02:45:08.055: INFO: rs-bz52n started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container donothing ready: false, restart count 0 Jan 22 02:45:08.055: INFO: pod-efed6c88-76fd-4889-8a42-f736d44b87a6 started at 2023-01-22 02:41:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: pod-subpath-test-inlinevolume-8c5p started at 2023-01-22 02:41:00 +0000 UTC (2+2 container statuses recorded) Jan 22 02:45:08.055: INFO: Init container init-volume-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Init container test-init-subpath-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-subpath-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-volume-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:08.055: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:08.055: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:08.055: INFO: indexed-job-3-p9nwq started at 2023-01-22 02:42:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container c ready: false, restart count 0 Jan 22 02:45:08.055: INFO: netserver-3 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container webserver ready: false, restart count 0 Jan 22 02:45:08.055: INFO: sample-webhook-deployment-6c69dbd86b-9ptj7 started at 2023-01-22 02:40:48 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container sample-webhook ready: false, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.055: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:08.055: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:08.055: INFO: pod-subpath-test-inlinevolume-rts6 started at 2023-01-22 02:40:49 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:08.055: INFO: Init container init-volume-inlinevolume-rts6 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: Container test-container-subpath-inlinevolume-rts6 ready: false, restart count 0 Jan 22 02:45:08.055: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:23 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:08.056: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:08.056: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:08.056: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:08.056: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:45:08.056: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:45:08.056: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:08.056: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:08.056: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.056: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.187: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:09.187: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:09.335: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 16931 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:45:09.335: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:09.486: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:09.645: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.645: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:45:09.645: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.645: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:45:09.646: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.646: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:45:09.646: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.646: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:45:09.646: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.646: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:45:09.646: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:09.646: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:09.646: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:09.646: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:09.646: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:45:09.646: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:09.646: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:45:09.646: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:45:09.646: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:45:09.646: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.646: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:45:09.646: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.646: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:45:10.124: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:10.124: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "fsgroupchangepolicy-9092" for this suite.
Find pod-c1563870-772d-4914-a1a9-6ae1736b51eb mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\smount\soptions$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Jan 22 02:34:13.619: Unexpected error: <*errors.errorString | 0xc0049f6060>: { s: "pod \"pod-80862d15-4984-4b9d-99fd-e96e46a3ac67\" is not Running: timed out waiting for the condition", } pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67" is not Running: timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418from junit_21.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:29:11.205: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should provision storage with mount options /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:180 Jan 22 02:29:12.254: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Jan 22 02:29:12.254: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Jan 22 02:29:12.254: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-1081-e2e-sc8thvn 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1081 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1081-e2e-sc8thvn,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1081 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1081-e2e-sc8thvn,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1081 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1081-e2e-sc8thvn,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-1081-e2e-sc8thvn 0decbe1e-3dcc-45fd-a194-1ea102516031 11270 0 2023-01-22 02:29:12 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2023-01-22 02:29:12 +0000 UTC FieldsV1 {"f:allowVolumeExpansion":{},"f:mountOptions":{},"f:parameters":{".":{},"f:encrypted":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:*Delete,MountOptions:[dirsync],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-gh8n9 pvc- provisioning-1081 1022e889-7a3a-434e-b16f-0afd50350ded 11272 0 2023-01-22 02:29:12 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2023-01-22 02:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1081-e2e-sc8thvn,VolumeMode:*Filesystem,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} Jan 22 02:34:13.619: FAIL: Unexpected error: <*errors.errorString | 0xc0049f6060>: { s: "pod \"pod-80862d15-4984-4b9d-99fd-e96e46a3ac67\" is not Running: timed out waiting for the condition", } pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67" is not Running: timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7938928, 0xc003551080}, 0xc0038a4800, 0xc000048000, 0xc000048d20, 0xc001095980, {0x0, 0x0}, {0x0, 0x0, ...}, ...}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:418 +0x807 k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:198 +0x3c5 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc000a3ad00, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f Jan 22 02:34:13.619: INFO: deleting claim "provisioning-1081"/"pvc-gh8n9" Jan 22 02:34:13.774: INFO: deleting storage class provisioning-1081-e2e-sc8thvn [AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "provisioning-1081". �[1mSTEP�[0m: Found 17 events. Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:12 +0000 UTC - event for pvc-gh8n9: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:13 +0000 UTC - event for pvc-gh8n9: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "provisioning-1081/pvc-gh8n9" Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:13 +0000 UTC - event for pvc-gh8n9: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:16 +0000 UTC - event for pvc-gh8n9: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-1022e889-7a3a-434e-b16f-0afd50350ded Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:17 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {default-scheduler } Scheduled: Successfully assigned provisioning-1081/pod-80862d15-4984-4b9d-99fd-e96e46a3ac67 to ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:19 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-1022e889-7a3a-434e-b16f-0afd50350ded" Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:20 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "70d6dd1b20207f8a91cb74a1ed2444c87b553ab19f8518df55d405b5af50cbbb" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:21 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:22 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "0109c4509851693d8bd15dae7ec68056f484e325f863deecdb79a8bce08f4d35" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:23 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "2b30cf69fd2e43e700084d9751981865e4c55a747ba129eb91b626cbd3e2c75d" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:24 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "747abcd7a4dcca938eb4c504baecc1ebd7aa1cb386c09a504430469dab065bc4" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:25 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "e00335b8eae762627ff54bc25859da835ffad74c099bf7b4e5b9ab4b5faf88f0" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:26 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9cd28f7e37cb3efe28c42589626fd36afb00aeddfa88173483d1142f596217a9" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:28 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "d183d90ad482a9c31e9c23ca0908fcb7d54686ddc989b2e3dc35179db3d9c6c4" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:29 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "23f7762c97341a2e233c56d71a76aab735c1acc3320b26683199c835bfa48e54" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:30 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "4562f18e4ee0eb2cf88ee341b3e48861d229383d27920c39ac812d47da2f7610" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.079: INFO: At 2023-01-22 02:29:31 +0000 UTC - event for pod-80862d15-4984-4b9d-99fd-e96e46a3ac67: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "0641039e1cfc6758d59125c871863d98afe3e6fa6067c2278c6493d4e99d4f25" network for pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67": networkPlugin cni failed to set up pod "pod-80862d15-4984-4b9d-99fd-e96e46a3ac67_provisioning-1081" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:14.229: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:34:14.229: INFO: pod-80862d15-4984-4b9d-99fd-e96e46a3ac67 ip-172-20-33-190.ap-northeast-1.compute.internal Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:29:17 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:29:17 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:29:17 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:29:17 +0000 UTC }] Jan 22 02:34:14.229: INFO: Jan 22 02:34:14.381: INFO: Unable to fetch provisioning-1081/pod-80862d15-4984-4b9d-99fd-e96e46a3ac67/write-pod logs: the server rejected our request for an unknown reason (get pods pod-80862d15-4984-4b9d-99fd-e96e46a3ac67) Jan 22 02:34:14.533: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:14.684: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 11676 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:29:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-22 02:29:19 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-019e723d932658f17 kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-019e723d932658f17,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4,DevicePath:,},},Config:nil,},} Jan 22 02:34:14.684: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:14.837: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:15.146: INFO: pod-80862d15-4984-4b9d-99fd-e96e46a3ac67 started at 2023-01-22 02:29:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:15.146: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:33:44 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:15.146: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:34:15.146: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:15.146: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:15.146: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:15.146: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:15.146: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:15.146: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: inline-volume-tester-rkzx7 started at 2023-01-22 02:29:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:15.146: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:30:13 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:15.146: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:34:15.146: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: test-rolling-update-deployment-8656fc4b57-c8wd7 started at 2023-01-22 02:30:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: netserver-0 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:15.146: INFO: test-rolling-update-controller-xhnb5 started at 2023-01-22 02:29:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container httpd ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container coredns ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.146: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:15.146: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:15.626: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:15.626: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:15.777: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 13224 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4110":"csi-mock-csi-mock-volumes-4110","ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:29:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:29:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3,DevicePath:,},},Config:nil,},} Jan 22 02:34:15.778: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:15.931: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:16.107: INFO: service-headless-xgkds started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:34:16.107: INFO: inline-volume-tester-7p6pc started at 2023-01-22 02:30:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:16.107: INFO: csi-mockplugin-0 started at 2023-01-22 02:29:11 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:16.107: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:16.107: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:34:16.107: INFO: Container mock ready: false, restart count 0 Jan 22 02:34:16.107: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-58nkp started at 2023-01-22 02:33:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:16.107: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.107: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.107: INFO: test-rolling-update-with-lb-7dcdf997cd-2nzq9 started at 2023-01-22 02:30:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:34:16.107: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.107: INFO: pod-configmaps-487886f1-48d2-404e-b45d-500ec77edc3f started at 2023-01-22 02:33:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container env-test ready: false, restart count 0 Jan 22 02:34:16.107: INFO: test-rolling-update-with-lb-79bf9f6988-rqqm8 started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:16.107: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:16.107: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.107: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.107: INFO: dns-test-7165a663-4d55-4543-acfd-a8c273537323 started at 2023-01-22 02:27:52 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:16.107: INFO: Container jessie-querier ready: false, restart count 0 Jan 22 02:34:16.107: INFO: Container querier ready: false, restart count 0 Jan 22 02:34:16.107: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:16.107: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:16.107: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:16.107: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:16.107: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:16.107: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.108: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.108: INFO: pvc-volume-tester-plxcq started at 2023-01-22 02:29:54 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container volume-tester ready: false, restart count 0 Jan 22 02:34:16.108: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:16.108: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:16.108: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.108: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.108: INFO: pod-subpath-test-preprovisionedpv-8dnv started at 2023-01-22 02:33:59 +0000 UTC (0+2 container statuses recorded) Jan 22 02:34:16.108: INFO: Container test-container-subpath-preprovisionedpv-8dnv ready: false, restart count 0 Jan 22 02:34:16.108: INFO: Container test-container-volume-preprovisionedpv-8dnv ready: false, restart count 0 Jan 22 02:34:16.108: INFO: pod-14b5f8c7-e30a-426e-8605-713988dc0cae started at 2023-01-22 02:34:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:16.108: INFO: metadata-volume-fd30831d-8c17-454a-91a4-d45d8348548b started at 2023-01-22 02:30:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container client-container ready: false, restart count 0 Jan 22 02:34:16.108: INFO: netserver-1 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:16.108: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-2tv2s started at 2023-01-22 02:33:55 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:16.108: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:16.108: INFO: csi-mockplugin-0 started at 2023-01-22 02:27:21 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:16.108: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:16.108: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:34:16.108: INFO: Container mock ready: true, restart count 0 Jan 22 02:34:16.108: INFO: csi-mockplugin-attacher-0 started at 2023-01-22 02:29:11 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:16.108: INFO: pod-projected-configmaps-12ebdd9f-aa1b-4d56-a05e-a8e067698121 started at 2023-01-22 02:29:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:16.108: INFO: Container projected-configmap-volume-test ready: false, restart count 0 Jan 22 02:34:16.601: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:16.601: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:16.752: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 12864 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8079":"ip-172-20-47-230.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:27:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:16.752: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:16.908: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:17.071: INFO: service-headless-p8n6b started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:34:17.071: INFO: pod-5e2f43d2-6145-4e2d-8712-db9b840587e7 started at 2023-01-22 02:30:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:17.071: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: test-rolling-update-with-lb-79bf9f6988-44b8c started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:17.071: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:27:08 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:17.071: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-dsw6d started at 2023-01-22 02:30:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:17.071: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container coredns ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: inline-volume-tester2-zmfh6 started at 2023-01-22 02:29:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:17.071: INFO: netserver-2 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:17.071: INFO: inline-volume-tester-6nngz started at 2023-01-22 02:27:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:17.071: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:17.071: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.071: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.071: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:18.096: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:18.096: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:18.246: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 13046 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:28:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:18.247: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:18.399: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:18.561: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: test-rolling-update-with-lb-79bf9f6988-4fp4m started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:18.562: INFO: pod-configmaps-220ddd82-1429-45eb-b1eb-7f5627bdf54e started at 2023-01-22 02:29:58 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:18.562: INFO: busybox-5e27475f-0f54-40db-a2ce-a1c9dee0380c started at 2023-01-22 02:32:42 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container busybox ready: true, restart count 0 Jan 22 02:34:18.562: INFO: netserver-3 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:18.562: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:18.562: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:18.562: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:18.562: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:18.562: INFO: service-headless-w4f7z started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container service-headless ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.562: INFO: pod-906bd654-ba45-4cea-8ad3-0065821bb928 started at 2023-01-22 02:33:49 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.562: INFO: Container test-container ready: false, restart count 0 Jan 22 02:34:19.266: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:19.266: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:19.417: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 12105 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:19.417: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:19.571: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:19.729: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:34:19.729: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:19.729: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:19.729: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:34:19.729: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:19.729: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:19.729: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:34:19.729: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:34:19.729: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:34:19.730: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:34:19.730: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:34:19.730: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:19.730: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:19.730: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:19.730: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:19.730: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:34:19.730: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:34:19.730: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:34:19.730: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:34:19.730: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:19.730: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:19.730: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.730: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:34:20.208: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:20.208: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "provisioning-1081" for this suite.
Find pod-80862d15-4984-4b9d-99fd-e96e46a3ac67 mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sdirectory\sspecified\sin\sthe\svolumeMount$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365 Jan 22 02:39:28.287: Unexpected error: <*errors.errorString | 0xc00567ebc0>: { s: "expected pod \"pod-subpath-test-dynamicpv-v99v\" success: Gave up after waiting 5m0s for pod \"pod-subpath-test-dynamicpv-v99v\" to be \"Succeeded or Failed\"", } expected pod "pod-subpath-test-dynamicpv-v99v" success: Gave up after waiting 5m0s for pod "pod-subpath-test-dynamicpv-v99v" to be "Succeeded or Failed" occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:770from junit_02.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:34:22.447: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should support readOnly directory specified in the volumeMount /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365 Jan 22 02:34:23.494: INFO: Creating resource for dynamic PV Jan 22 02:34:23.494: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass provisioning-2350-e2e-sc2znb5 �[1mSTEP�[0m: creating a claim Jan 22 02:34:23.645: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: Creating pod pod-subpath-test-dynamicpv-v99v �[1mSTEP�[0m: Creating a pod to test subpath Jan 22 02:34:24.097: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-v99v" in namespace "provisioning-2350" to be "Succeeded or Failed" Jan 22 02:34:24.247: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 149.744242ms Jan 22 02:34:26.399: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2.301423447s Jan 22 02:34:28.549: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4.451309318s Jan 22 02:34:30.700: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 6.602225538s Jan 22 02:34:32.851: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 8.753255941s Jan 22 02:34:35.001: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 10.903970451s Jan 22 02:34:37.152: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 13.054849533s Jan 22 02:34:39.304: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 15.206834938s Jan 22 02:34:41.454: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 17.356826982s Jan 22 02:34:43.604: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 19.506931817s Jan 22 02:34:45.756: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 21.658516081s Jan 22 02:34:47.906: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 23.80852943s Jan 22 02:34:50.056: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 25.959130977s Jan 22 02:34:52.207: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 28.10924897s Jan 22 02:34:54.357: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 30.259316974s Jan 22 02:34:56.507: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 32.410129574s Jan 22 02:34:58.657: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 34.559959158s Jan 22 02:35:00.808: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 36.710744738s Jan 22 02:35:02.959: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 38.861470957s Jan 22 02:35:05.109: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 41.011708299s Jan 22 02:35:07.261: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 43.163338968s Jan 22 02:35:09.411: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 45.314019947s Jan 22 02:35:11.561: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 47.463838173s Jan 22 02:35:13.712: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 49.614280825s Jan 22 02:35:15.861: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 51.763928269s Jan 22 02:35:18.011: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 53.913670212s Jan 22 02:35:20.161: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 56.06410733s Jan 22 02:35:22.311: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 58.214111205s Jan 22 02:35:24.461: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.363756662s Jan 22 02:35:26.612: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.514909973s Jan 22 02:35:28.763: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.665887541s Jan 22 02:35:30.914: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.817072819s Jan 22 02:35:33.065: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.96807004s Jan 22 02:35:35.216: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m11.118292565s Jan 22 02:35:37.370: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m13.272321778s Jan 22 02:35:39.520: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m15.422853396s Jan 22 02:35:41.670: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m17.572816469s Jan 22 02:35:43.820: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m19.723047462s Jan 22 02:35:45.970: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m21.873129409s Jan 22 02:35:48.131: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.033723616s Jan 22 02:35:50.282: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.184535349s Jan 22 02:35:52.432: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.334697593s Jan 22 02:35:54.583: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.48548207s Jan 22 02:35:56.737: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.639811423s Jan 22 02:35:58.887: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.789624369s Jan 22 02:36:01.038: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.940596332s Jan 22 02:36:03.188: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m39.090717701s Jan 22 02:36:05.339: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m41.241569964s Jan 22 02:36:07.490: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m43.392257134s Jan 22 02:36:09.640: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m45.542493545s Jan 22 02:36:11.791: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m47.693707426s Jan 22 02:36:13.941: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m49.843813089s Jan 22 02:36:16.092: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m51.994886539s Jan 22 02:36:18.243: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.145712579s Jan 22 02:36:20.394: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.296404775s Jan 22 02:36:22.544: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.446329781s Jan 22 02:36:24.695: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.597594057s Jan 22 02:36:26.846: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.749054127s Jan 22 02:36:28.996: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.898857214s Jan 22 02:36:31.146: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m7.048821091s Jan 22 02:36:33.297: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m9.199462978s Jan 22 02:36:35.447: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m11.349796304s Jan 22 02:36:37.598: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m13.500402404s Jan 22 02:36:39.748: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m15.650645507s Jan 22 02:36:41.899: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m17.801589734s Jan 22 02:36:44.049: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m19.95197031s Jan 22 02:36:46.200: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.103104687s Jan 22 02:36:48.351: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.253485237s Jan 22 02:36:50.502: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.404524995s Jan 22 02:36:52.653: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.55551285s Jan 22 02:36:54.804: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.706705534s Jan 22 02:36:56.955: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.857900916s Jan 22 02:36:59.107: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m35.009608426s Jan 22 02:37:01.258: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m37.160827122s Jan 22 02:37:03.409: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m39.311642557s Jan 22 02:37:05.560: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m41.462774162s Jan 22 02:37:07.711: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m43.613639878s Jan 22 02:37:09.861: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m45.763643566s Jan 22 02:37:12.012: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m47.914889216s Jan 22 02:37:14.162: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.06518919s Jan 22 02:37:16.313: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.216130907s Jan 22 02:37:18.464: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.366326141s Jan 22 02:37:20.614: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.516638068s Jan 22 02:37:22.764: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.666830384s Jan 22 02:37:24.914: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.816763638s Jan 22 02:37:27.064: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.966979125s Jan 22 02:37:29.214: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m5.117124923s Jan 22 02:37:31.366: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m7.268435839s Jan 22 02:37:33.516: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m9.418781755s Jan 22 02:37:35.667: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m11.569725516s Jan 22 02:37:37.817: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m13.719924171s Jan 22 02:37:39.967: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m15.870159461s Jan 22 02:37:42.117: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.020105152s Jan 22 02:37:44.267: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.170035116s Jan 22 02:37:46.418: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.321118201s Jan 22 02:37:48.569: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.471939716s Jan 22 02:37:50.721: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.623592187s Jan 22 02:37:52.871: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.773228448s Jan 22 02:37:55.021: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.923347097s Jan 22 02:37:57.171: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m33.073470335s Jan 22 02:37:59.321: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m35.223583102s Jan 22 02:38:01.472: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m37.37465553s Jan 22 02:38:03.623: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m39.525369177s Jan 22 02:38:05.773: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m41.675872236s Jan 22 02:38:07.923: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m43.82580493s Jan 22 02:38:10.074: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m45.976460778s Jan 22 02:38:12.224: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.126199854s Jan 22 02:38:14.373: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.276157082s Jan 22 02:38:16.525: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.427330834s Jan 22 02:38:18.676: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.578600085s Jan 22 02:38:20.826: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.728875542s Jan 22 02:38:22.977: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.880160456s Jan 22 02:38:25.129: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m1.031697455s Jan 22 02:38:27.279: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m3.181717812s Jan 22 02:38:29.429: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m5.331581828s Jan 22 02:38:31.580: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m7.482945648s Jan 22 02:38:33.732: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m9.634307794s Jan 22 02:38:35.883: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m11.785725847s Jan 22 02:38:38.033: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m13.935247064s Jan 22 02:38:40.184: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.086412455s Jan 22 02:38:42.334: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.237189979s Jan 22 02:38:44.485: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.387486929s Jan 22 02:38:46.635: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.537756527s Jan 22 02:38:48.785: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.687729377s Jan 22 02:38:50.936: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.838472954s Jan 22 02:38:53.087: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.989235231s Jan 22 02:38:55.239: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m31.141410584s Jan 22 02:38:57.390: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m33.292257794s Jan 22 02:38:59.541: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m35.443651906s Jan 22 02:39:01.691: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m37.593702076s Jan 22 02:39:03.841: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m39.743610512s Jan 22 02:39:05.992: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m41.894883323s Jan 22 02:39:08.142: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.044997324s Jan 22 02:39:10.292: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.195087247s Jan 22 02:39:12.443: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.34598845s Jan 22 02:39:14.594: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.496303813s Jan 22 02:39:16.743: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.646125116s Jan 22 02:39:18.893: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.795764667s Jan 22 02:39:21.043: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.945537304s Jan 22 02:39:23.193: INFO: Pod "pod-subpath-test-dynamicpv-v99v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m59.096129408s Jan 22 02:39:25.513: INFO: Failed to get logs from node "ip-172-20-39-135.ap-northeast-1.compute.internal" pod "pod-subpath-test-dynamicpv-v99v" container "init-volume-dynamicpv-v99v": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-v99v) Jan 22 02:39:25.666: INFO: Failed to get logs from node "ip-172-20-39-135.ap-northeast-1.compute.internal" pod "pod-subpath-test-dynamicpv-v99v" container "test-init-volume-dynamicpv-v99v": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-v99v) Jan 22 02:39:25.835: INFO: Failed to get logs from node "ip-172-20-39-135.ap-northeast-1.compute.internal" pod "pod-subpath-test-dynamicpv-v99v" container "test-container-subpath-dynamicpv-v99v": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-v99v) �[1mSTEP�[0m: delete the pod Jan 22 02:39:25.988: INFO: Waiting for pod pod-subpath-test-dynamicpv-v99v to disappear Jan 22 02:39:26.138: INFO: Pod pod-subpath-test-dynamicpv-v99v still exists Jan 22 02:39:28.138: INFO: Waiting for pod pod-subpath-test-dynamicpv-v99v to disappear Jan 22 02:39:28.287: INFO: Pod pod-subpath-test-dynamicpv-v99v no longer exists Jan 22 02:39:28.287: FAIL: Unexpected error: <*errors.errorString | 0xc00567ebc0>: { s: "expected pod \"pod-subpath-test-dynamicpv-v99v\" success: Gave up after waiting 5m0s for pod \"pod-subpath-test-dynamicpv-v99v\" to be \"Succeeded or Failed\"", } expected pod "pod-subpath-test-dynamicpv-v99v" success: Gave up after waiting 5m0s for pod "pod-subpath-test-dynamicpv-v99v" to be "Succeeded or Failed" occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc002e75400?, {0x705074c?, 0x0?}, 0xc002e75400, 0x0, {0xc00559f118, 0x1, 0x1}, 0x34?) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:770 +0x176 k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:567 k8s.io/kubernetes/test/e2e/storage/testsuites.testReadFile(0xc003659080, {0xc005670f30, 0x16}, 0xc002e75400, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:691 +0x15c k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func16() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:377 +0x1da k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc000d084e0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f �[1mSTEP�[0m: Deleting pod Jan 22 02:39:28.288: INFO: Deleting pod "pod-subpath-test-dynamicpv-v99v" in namespace "provisioning-2350" �[1mSTEP�[0m: Deleting pvc Jan 22 02:39:28.736: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.combzxzw" Jan 22 02:39:28.886: INFO: Waiting up to 3m0s for PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb to get deleted Jan 22 02:39:29.036: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (149.127567ms) Jan 22 02:39:34.186: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (5.299063809s) Jan 22 02:39:39.336: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (10.449611229s) Jan 22 02:39:44.487: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (15.600492031s) Jan 22 02:39:49.638: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (20.751005851s) Jan 22 02:39:54.788: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (25.901566921s) Jan 22 02:39:59.938: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb found and phase=Released (31.051218827s) Jan 22 02:40:05.087: INFO: PersistentVolume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb was removed �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "provisioning-2350". �[1mSTEP�[0m: Found 17 events. Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:23 +0000 UTC - event for ebs.csi.aws.combzxzw: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:24 +0000 UTC - event for ebs.csi.aws.combzxzw: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:24 +0000 UTC - event for ebs.csi.aws.combzxzw: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "provisioning-2350/ebs.csi.aws.combzxzw" Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:27 +0000 UTC - event for ebs.csi.aws.combzxzw: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:28 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {default-scheduler } Scheduled: Successfully assigned provisioning-2350/pod-subpath-test-dynamicpv-v99v to ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:30 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-c4366841-0e91-48a6-9ef0-c52b4de7bfeb" Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:33 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "4cd37dfcf69547f760836a22c010575713435fc40f48588415001866761b2c89" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:35 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:37 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "82c01775dd6df8a683e11b55bd2f2ee6c3bb28295af01bee9e3dff3469a83ec3" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:41 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "5cdb71254ad9a6e1c6430ca9465fd308b533827f6ec8392205fdab7b893ad569" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:48 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9ba4722afc4e19d3f78bc17804edfee4091e0a4faa8aeae584c90da631fee274" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:51 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "3488bd078ff7e165cf7fd2b3009dc2f8f49429b258306785a38bce36e212b9b0" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:54 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "43ff1e6fe491c7eac772631810f4b2aaa83d7d41dd9b848135c564a71898364c" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:34:57 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1514b0574fb5b0435b964b205a7a5902b1a0a7fb63a0055d1b395bb2a0dd25ba" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:35:01 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "fb33efc1f301fc93c97b8a9cae36173b0c63fe9e08088c481e875af4452a4738" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:35:06 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ad29d7deb9dc3beba8e07397e3371f40d37724d29e479b1db656d9c4dfdb31e9" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.389: INFO: At 2023-01-22 02:35:09 +0000 UTC - event for pod-subpath-test-dynamicpv-v99v: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "07e997fcae607b837d6bd421612f1317ccc0304503fc262ef3919be29cad6a09" network for pod "pod-subpath-test-dynamicpv-v99v": networkPlugin cni failed to set up pod "pod-subpath-test-dynamicpv-v99v_provisioning-2350" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:40:05.538: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:40:05.538: INFO: Jan 22 02:40:05.690: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:40:05.839: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 14150 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-949":"ip-172-20-33-190.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:29:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-22 02:29:19 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:35:27 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:35:27 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:35:27 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:35:27 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-949^6a90a474-99fd-11ed-a63d-6e45846a2bf5 kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-949^6a90a474-99fd-11ed-a63d-6e45846a2bf5,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4,DevicePath:,},},Config:nil,},} Jan 22 02:40:05.840: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:40:05.992: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:40:06.155: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-cl8kh started at 2023-01-22 02:35:12 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.155: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:40:06.155: INFO: pod-24706ce3-52ee-433b-90d6-e2de4ae435c3 started at 2023-01-22 02:35:29 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.155: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:40:06.155: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.155: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.155: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:40:06.155: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:40:06.155: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:40:06.155: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:40:06.155: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:40:06.156: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:06.156: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:40:06.156: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.156: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.156: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.156: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-jbdfh started at 2023-01-22 02:35:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:40:06.156: INFO: local-injector started at 2023-01-22 02:35:28 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container local-injector ready: false, restart count 0 Jan 22 02:40:06.156: INFO: inline-volume-tester-rkzx7 started at 2023-01-22 02:29:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:40:06.156: INFO: rs-tcq62 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Container donothing ready: false, restart count 0 Jan 22 02:40:06.156: INFO: pod-subpath-test-preprovisionedpv-frxb started at 2023-01-22 02:35:29 +0000 UTC (2+1 container statuses recorded) Jan 22 02:40:06.156: INFO: Init container init-volume-preprovisionedpv-frxb ready: false, restart count 0 Jan 22 02:40:06.157: INFO: Init container test-init-volume-preprovisionedpv-frxb ready: false, restart count 0 Jan 22 02:40:06.157: INFO: Container test-container-subpath-preprovisionedpv-frxb ready: false, restart count 0 Jan 22 02:40:06.157: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.157: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.157: INFO: pod-subpath-test-preprovisionedpv-jnl2 started at 2023-01-22 02:35:44 +0000 UTC (2+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Init container init-volume-preprovisionedpv-jnl2 ready: false, restart count 0 Jan 22 02:40:06.157: INFO: Init container test-init-volume-preprovisionedpv-jnl2 ready: false, restart count 0 Jan 22 02:40:06.157: INFO: Container test-container-subpath-preprovisionedpv-jnl2 ready: false, restart count 0 Jan 22 02:40:06.157: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.157: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.157: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-xvgw4 started at 2023-01-22 02:35:18 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:40:06.157: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.157: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.157: INFO: Container coredns ready: true, restart count 0 Jan 22 02:40:06.157: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:36:13 +0000 UTC (0+7 container statuses recorded) Jan 22 02:40:06.157: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:40:06.158: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.158: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:06.158: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-bjhk6 started at 2023-01-22 02:35:29 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.158: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:40:06.158: INFO: netserver-0 started at 2023-01-22 02:36:07 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:06.158: INFO: Container webserver ready: false, restart count 0 Jan 22 02:40:06.158: INFO: pod-subpath-test-dynamicpv-r8v7 started at 2023-01-22 02:35:22 +0000 UTC (1+1 container statuses recorded) Jan 22 02:40:06.158: INFO: Init container init-volume-dynamicpv-r8v7 ready: false, restart count 0 Jan 22 02:40:06.158: INFO: Container test-container-subpath-dynamicpv-r8v7 ready: false, restart count 0 Jan 22 02:40:06.158: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:33:44 +0000 UTC (0+7 container statuses recorded) Jan 22 02:40:06.158: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:40:06.158: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:40:06.158: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:40:06.158: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:40:06.158: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:40:06.158: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:06.158: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:40:06.772: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:40:06.773: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:40:06.923: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 15982 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:29:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:29:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:39:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:39:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:39:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:39:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-001435dfbafa8afcd kubernetes.io/csi/ebs.csi.aws.com^vol-02f9a67e9fa46c78f kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3 kubernetes.io/csi/ebs.csi.aws.com^vol-081a404f339e0b081],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-081a404f339e0b081,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02f9a67e9fa46c78f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-001435dfbafa8afcd,DevicePath:,},},Config:nil,},} Jan 22 02:40:06.925: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:40:07.077: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: netserver-1 started at 2023-01-22 02:36:07 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container webserver ready: true, restart count 0 Jan 22 02:40:07.234: INFO: pod-c1563870-772d-4914-a1a9-6ae1736b51eb started at 2023-01-22 02:37:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: service-headless-xgkds started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container service-headless ready: true, restart count 0 Jan 22 02:40:07.234: INFO: inline-volume-tester-7p6pc started at 2023-01-22 02:30:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: pod-36df73d8-2733-4989-9392-bfab945018c2 started at 2023-01-22 02:37:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:40:07.234: INFO: hostpathsymlink-injector started at 2023-01-22 02:35:58 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container hostpathsymlink-injector ready: false, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: inline-volume-tester-mvj8p started at 2023-01-22 02:35:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:40:07.234: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:40:07.234: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:07.234: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: rs-sp685 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container donothing ready: false, restart count 0 Jan 22 02:40:07.234: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:07.234: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:40:07.234: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:40:07.234: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:40:07.720: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:40:07.720: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:40:07.870: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 15739 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:27:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:38:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:38:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:38:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:38:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:40:07.870: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:40:08.022: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:40:08.180: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-h4m4v started at 2023-01-22 02:36:10 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:40:08.180: INFO: service-headless-p8n6b started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:40:08.180: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:40:08.180: INFO: netserver-2 started at 2023-01-22 02:36:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container webserver ready: true, restart count 0 Jan 22 02:40:08.180: INFO: pod-946d35f7-8930-48b4-b51b-a546640b26e5 started at 2023-01-22 02:36:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:40:08.180: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.180: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.180: INFO: Container coredns ready: true, restart count 0 Jan 22 02:40:08.180: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:40:08.181: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.181: INFO: local-injector started at 2023-01-22 02:35:29 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container local-injector ready: false, restart count 0 Jan 22 02:40:08.181: INFO: csi-mockplugin-attacher-0 started at 2023-01-22 02:35:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:40:08.181: INFO: hostpath-symlink-prep-provisioning-6067 started at 2023-01-22 02:36:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container init-volume-provisioning-6067 ready: false, restart count 0 Jan 22 02:40:08.181: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.181: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-jspp7 started at 2023-01-22 02:35:05 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:40:08.181: INFO: csi-mockplugin-0 started at 2023-01-22 02:35:15 +0000 UTC (0+3 container statuses recorded) Jan 22 02:40:08.181: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:40:08.181: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:40:08.181: INFO: Container mock ready: false, restart count 0 Jan 22 02:40:08.181: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:08.181: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:08.181: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:40:08.181: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:40:08.181: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:08.181: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:40:09.181: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:40:09.181: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:40:09.331: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 15777 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:38:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:38:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:38:45 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:38:45 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:38:45 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:38:45 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8,DevicePath:,},},Config:nil,},} Jan 22 02:40:09.332: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:40:09.483: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: netserver-3 started at 2023-01-22 02:36:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container webserver ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: metadata-volume-fd1ed2a6-ac30-400c-b47f-70e2211b5282 started at 2023-01-22 02:36:07 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container client-container ready: false, restart count 0 Jan 22 02:40:09.644: INFO: sample-apiserver-deployment-7cdc9f5bf7-6z8ps started at 2023-01-22 02:36:16 +0000 UTC (0+2 container statuses recorded) Jan 22 02:40:09.644: INFO: Container etcd ready: false, restart count 0 Jan 22 02:40:09.644: INFO: Container sample-apiserver ready: false, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: ss-0 started at 2023-01-22 02:36:26 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container webserver ready: false, restart count 0 Jan 22 02:40:09.644: INFO: rs-bz52n started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container donothing ready: false, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:40:09.644: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:40:09.644: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:09.644: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:40:09.644: INFO: pod-9550b8c9-bbad-48e8-aed5-be8f0beef3f3 started at 2023-01-22 02:38:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:40:09.644: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:40:09.644: INFO: service-headless-w4f7z started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container service-headless ready: true, restart count 0 Jan 22 02:40:09.644: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:09.644: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:40:10.359: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:40:10.359: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:40:10.509: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 14758 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:36:06 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:36:06 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:36:06 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:36:06 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:40:10.509: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:40:10.666: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:40:10.823: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:40:10.823: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:40:10.823: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:40:10.823: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:40:10.823: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:40:10.823: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:40:10.823: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:40:10.823: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:40:10.823: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:40:10.823: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:40:10.823: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:40:10.823: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:40:10.823: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:40:10.823: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:40:10.823: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:40:10.823: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:40:11.305: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:40:11.305: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "provisioning-2350" for this suite.
Find pod-subpath-test-dynamicpv-v99v mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(delayed\sbinding\)\]\stopology\sshould\sprovision\sa\svolume\sand\sschedule\sa\spod\swith\sAllowedTopologies$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:164 Jan 22 02:46:42.815: Unexpected error: <*errors.errorString | 0xc00025c240>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:180from junit_24.xml
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:41:40.720: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename topology �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should provision a volume and schedule a pod with AllowedTopologies /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:164 Jan 22 02:41:41.911: INFO: found topology map[topology.ebs.csi.aws.com/zone:ap-northeast-1a] Jan 22 02:41:41.912: INFO: Creating storage class object and pvc object for driver - sc: &StorageClass{ObjectMeta:{topology-1862-e2e-sctxtvz 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{{[{topology.ebs.csi.aws.com/zone [ap-northeast-1a]}]},},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- topology-1862 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*topology-1862-e2e-sctxtvz,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: Creating sc �[1mSTEP�[0m: Creating pvc �[1mSTEP�[0m: Creating pod Jan 22 02:46:42.815: FAIL: Unexpected error: <*errors.errorString | 0xc00025c240>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*topologyTestSuite).DefineTests.func3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:180 +0x4cb k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc0001a3860, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f �[1mSTEP�[0m: Deleting pod Jan 22 02:46:42.815: INFO: Deleting pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e" in namespace "topology-1862" Jan 22 02:46:42.967: INFO: Wait up to 5m0s for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e" to be fully deleted �[1mSTEP�[0m: Deleting pvc Jan 22 02:46:57.562: INFO: Deleting PersistentVolumeClaim "pvc-bv5xp" Jan 22 02:46:57.725: INFO: Waiting up to 3m0s for PersistentVolume pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69 to get deleted Jan 22 02:46:57.875: INFO: PersistentVolume pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69 found and phase=Released (149.491913ms) Jan 22 02:47:03.023: INFO: PersistentVolume pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69 found and phase=Released (5.2982189s) Jan 22 02:47:08.173: INFO: PersistentVolume pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69 found and phase=Released (10.447750592s) Jan 22 02:47:13.323: INFO: PersistentVolume pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69 was removed �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "topology-1862". �[1mSTEP�[0m: Found 17 events. Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:42 +0000 UTC - event for pvc-bv5xp: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:42 +0000 UTC - event for pvc-bv5xp: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "topology-1862/pvc-bv5xp" Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:42 +0000 UTC - event for pvc-bv5xp: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:45 +0000 UTC - event for pvc-bv5xp: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69 Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:46 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {default-scheduler } Scheduled: Successfully assigned topology-1862/pod-72c4e40c-da11-4c01-bd78-334fabe5b47e to ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:48 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-f62957ea-0dd0-4434-aed2-e73771cd5e69" Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:55 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "afaa7c455d4ca7dae04b3da9db4c3a7c0c9facec7fbb2bd27d0d9ab4e39ce16b" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:56 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:47:13.624: INFO: At 2023-01-22 02:41:57 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9b7e94f220ac1e087e929c3d6ab9812c7c2e0c3f6c412656380453c9849f707a" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.624: INFO: At 2023-01-22 02:42:00 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "2274da304c369599e9ce0174927955bf63dbf9c0bcc121e52c33c18f4e899fd8" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.624: INFO: At 2023-01-22 02:42:02 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "798072a977b1ef1486079209e454c0bd648864a6b8c9272b2b6139237f239f2e" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.625: INFO: At 2023-01-22 02:42:03 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "e504c47051bb38fc1532ed0e9860d9afd501a2cc95744cc5a5bc531303fd7ad2" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.625: INFO: At 2023-01-22 02:42:05 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "632ff1978d673e9b876f33bb3c21773ddb98cccd98ba076fc834bf9c5274b405" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.625: INFO: At 2023-01-22 02:42:07 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "5f86ff0b444a226a6b408684ac8845ab1b193107c1e0a32a1f7b5659a3936fe6" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.625: INFO: At 2023-01-22 02:42:10 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "72e74abce774132a5fac293f9400457a2999d57d4ca647e04dbd046db4b4eaf5" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.625: INFO: At 2023-01-22 02:42:12 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "71cd6473f5a4aecbad651260eca8b20ba1231fc397b9ac9a4ff655760b66fdfc" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.625: INFO: At 2023-01-22 02:42:15 +0000 UTC - event for pod-72c4e40c-da11-4c01-bd78-334fabe5b47e: {kubelet ip-172-20-47-230.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "6edef23602fc90ce29058a01266856912fd4c6713bdc1904da3d01e6051fe790" network for pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e": networkPlugin cni failed to set up pod "pod-72c4e40c-da11-4c01-bd78-334fabe5b47e_topology-1862" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:47:13.774: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:47:13.774: INFO: Jan 22 02:47:13.927: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:47:14.079: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 19170 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-01-22 02:44:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:47:14.079: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:47:14.230: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:47:14.385: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container coredns ready: true, restart count 0 Jan 22 02:47:14.385: INFO: netserver-0 started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container webserver ready: false, restart count 0 Jan 22 02:47:14.385: INFO: test-webserver-5acccf8f-d6d3-4300-b3c9-919af29118b8 started at 2023-01-22 02:47:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container test-webserver ready: false, restart count 0 Jan 22 02:47:14.385: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:47:14.385: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:47:14.385: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:14.385: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:47:14.385: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:14.385: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:47:14.385: INFO: pod-f8edc88c-9fb1-4563-8ca9-d16fb48469d4 started at 2023-01-22 02:47:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:47:14.385: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-56wjc started at 2023-01-22 02:47:00 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:14.385: INFO: pod-4c9ec1a5-026f-4951-86ea-c92107e0e121 started at 2023-01-22 02:47:04 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container write-pod ready: true, restart count 0 Jan 22 02:47:14.385: INFO: server started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:14.385: INFO: pod-3bc6ae18-aa4b-4beb-ba3f-128f64219c17 started at <nil> (0+0 container statuses recorded) Jan 22 02:47:14.385: INFO: update-demo-nautilus-fnpl8 started at 2023-01-22 02:47:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container update-demo ready: false, restart count 0 Jan 22 02:47:14.385: INFO: pod-secrets-ba450992-9309-4c59-9916-fae6c7689e65 started at 2023-01-22 02:47:10 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:14.385: INFO: Container secret-volume-test ready: false, restart count 0 Jan 22 02:47:14.868: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:47:14.868: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:47:15.017: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 20628 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-39-135.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8815":"ip-172-20-39-135.ap-northeast-1.compute.internal","csi-mock-csi-mock-volumes-1842":"csi-mock-csi-mock-volumes-1842","csi-mock-csi-mock-volumes-2041":"csi-mock-csi-mock-volumes-2041","ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-01-22 02:47:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:08 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:08 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:08 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:47:08 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:47:15.017: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:47:15.168: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:47:15.339: INFO: csi-mockplugin-resizer-0 started at 2023-01-22 02:42:35 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:15.339: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:47:15.339: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:47:15.339: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:47:15.339: INFO: csi-mockplugin-attacher-0 started at 2023-01-22 02:46:12 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:15.339: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:47:15.339: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:10 +0000 UTC (0+7 container statuses recorded) Jan 22 02:47:15.339: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:47:15.339: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:47:15.339: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:47:15.339: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:47:15.339: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:47:15.339: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:47:15.339: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:47:15.339: INFO: inline-volume-tester-9rmrs started at 2023-01-22 02:41:10 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:15.339: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:47:15.339: INFO: csi-mockplugin-0 started at 2023-01-22 02:47:05 +0000 UTC (0+4 container statuses recorded) Jan 22 02:47:15.339: INFO: Container busybox ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container mock ready: true, restart count 0 Jan 22 02:47:15.339: INFO: webserver-pod started at 2023-01-22 02:46:52 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:15.339: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:47:15.339: INFO: csi-mockplugin-0 started at 2023-01-22 02:42:35 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:15.339: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container mock ready: true, restart count 0 Jan 22 02:47:15.339: INFO: netserver-1 started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:15.339: INFO: Container webserver ready: false, restart count 0 Jan 22 02:47:15.339: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:15.339: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:47:15.339: INFO: csi-mockplugin-0 started at 2023-01-22 02:46:11 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:15.339: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:47:15.339: INFO: Container mock ready: true, restart count 0 Jan 22 02:47:15.831: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:47:15.831: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:47:15.980: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 20216 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:00 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:00 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:47:00 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:47:00 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:47:15.980: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:47:16.135: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:47:16.290: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:47:16.290: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:47:16.290: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:47:16.290: INFO: pod-subpath-test-preprovisionedpv-7rn5 started at 2023-01-22 02:47:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Init container init-volume-preprovisionedpv-7rn5 ready: false, restart count 0 Jan 22 02:47:16.290: INFO: Container test-container-subpath-preprovisionedpv-7rn5 ready: false, restart count 0 Jan 22 02:47:16.290: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Container coredns ready: true, restart count 0 Jan 22 02:47:16.290: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-fmm88 started at 2023-01-22 02:47:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:16.290: INFO: update-demo-nautilus-66g8w started at 2023-01-22 02:47:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Container update-demo ready: true, restart count 0 Jan 22 02:47:16.290: INFO: rs-qcs8g started at 2023-01-22 02:47:00 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Container donothing ready: false, restart count 0 Jan 22 02:47:16.290: INFO: pod-projected-secrets-a47240b4-3d7a-49b1-8bb6-26f3abe27f43 started at 2023-01-22 02:47:07 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.290: INFO: Container projected-secret-volume-test ready: false, restart count 0 Jan 22 02:47:16.291: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-nsx6w started at 2023-01-22 02:47:12 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.291: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:16.291: INFO: netserver-2 started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.291: INFO: Container webserver ready: false, restart count 0 Jan 22 02:47:16.291: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:16.291: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:47:16.291: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:16.291: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:47:16.291: INFO: tester started at 2023-01-22 02:47:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:16.291: INFO: Container tester ready: true, restart count 0 Jan 22 02:47:16.785: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:47:16.786: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:47:16.934: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 20926 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-7219":"ip-172-20-54-133.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:38:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:38:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:46:37 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-7219^13c03b5b-99ff-11ed-aff9-f630865f8ebe,DevicePath:,},},Config:nil,},} Jan 22 02:47:16.935: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:47:17.086: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:47:17.251: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:47:10 +0000 UTC (0+7 container statuses recorded) Jan 22 02:47:17.251: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:47:17.251: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:47:17.251: INFO: busybox-3fe937d3-a40e-486d-862f-80400141c5b3 started at 2023-01-22 02:46:36 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container busybox ready: true, restart count 0 Jan 22 02:47:17.251: INFO: hostexec-ip-172-20-54-133.ap-northeast-1.compute.internal-nd24z started at 2023-01-22 02:47:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:17.251: INFO: hostexec-ip-172-20-54-133.ap-northeast-1.compute.internal-5mwcm started at 2023-01-22 02:46:57 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:17.251: INFO: pod-61c122db-b5b7-46a8-b40d-c9c362dd8278 started at 2023-01-22 02:47:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:47:17.251: INFO: pod-efed6c88-76fd-4889-8a42-f736d44b87a6 started at 2023-01-22 02:41:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container write-pod ready: true, restart count 0 Jan 22 02:47:17.251: INFO: pause-pod-1 started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:47:17.251: INFO: netserver-3 started at 2023-01-22 02:47:01 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:17.251: INFO: Container webserver ready: false, restart count 0 Jan 22 02:47:17.251: INFO: pod-subpath-test-preprovisionedpv-2x6f started at 2023-01-22 02:47:14 +0000 UTC (1+2 container statuses recorded) Jan 22 02:47:17.251: INFO: Init container test-init-subpath-preprovisionedpv-2x6f ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container test-container-subpath-preprovisionedpv-2x6f ready: false, restart count 0 Jan 22 02:47:17.251: INFO: Container test-container-volume-preprovisionedpv-2x6f ready: false, restart count 0 Jan 22 02:47:17.251: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:17.251: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:17.251: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:47:17.731: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:47:17.731: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:47:17.880: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 19078 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:46:18 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:47:17.881: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:47:18.032: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:47:18.189: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:47:18.189: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:18.189: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:47:18.189: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:47:18.189: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:47:18.189: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:47:18.189: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:47:18.189: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:47:18.189: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:47:18.189: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:47:18.189: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:47:18.189: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:47:18.189: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:47:18.189: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:47:18.189: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:47:18.189: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:47:18.662: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:47:18.662: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "topology-1862" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(filesystem\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Jan 22 02:34:05.954: Unexpected error: <*errors.errorString | 0xc000232230>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385from junit_02.xml
[BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:29:03.826: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename volumemode �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should not mount / map unused volumes in a pod [LinuxOnly] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:352 Jan 22 02:29:04.890: INFO: Creating resource for dynamic PV Jan 22 02:29:04.890: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass volumemode-1892-e2e-schbjqv �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Creating pod Jan 22 02:34:05.954: FAIL: Unexpected error: <*errors.errorString | 0xc000232230>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:385 +0x62b k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc000d084e0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f Jan 22 02:34:05.954: INFO: Deleting pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e" in namespace "volumemode-1892" Jan 22 02:34:06.107: INFO: Wait up to 5m0s for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e" to be fully deleted �[1mSTEP�[0m: Deleting pvc Jan 22 02:34:10.705: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comt4c6h" Jan 22 02:34:10.857: INFO: Waiting up to 3m0s for PersistentVolume pvc-30296cd9-70d7-4c9c-bb8e-56d4d9acfe8e to get deleted Jan 22 02:34:11.007: INFO: PersistentVolume pvc-30296cd9-70d7-4c9c-bb8e-56d4d9acfe8e found and phase=Released (149.117376ms) Jan 22 02:34:16.156: INFO: PersistentVolume pvc-30296cd9-70d7-4c9c-bb8e-56d4d9acfe8e was removed �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "volumemode-1892". �[1mSTEP�[0m: Found 17 events. Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:05 +0000 UTC - event for ebs.csi.aws.comt4c6h: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:05 +0000 UTC - event for ebs.csi.aws.comt4c6h: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:05 +0000 UTC - event for ebs.csi.aws.comt4c6h: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "volumemode-1892/ebs.csi.aws.comt4c6h" Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:08 +0000 UTC - event for ebs.csi.aws.comt4c6h: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-30296cd9-70d7-4c9c-bb8e-56d4d9acfe8e Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:09 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {default-scheduler } Scheduled: Successfully assigned volumemode-1892/pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e to ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:10 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "26f4af440cf63d18ceca6296c28de0d8f6189cd2ecb3b81004fc6c6050634680" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:10 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:11 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-30296cd9-70d7-4c9c-bb8e-56d4d9acfe8e" Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:12 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "d7fc44b6f4ad20f401e64e409b7b3f559eaa9c557a50e0c65553ab6c156650d1" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:16 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "736fdde23304a183e759d932f676a7bb1941d27a5d47978e36bab82ed48a5179" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:20 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9a71551388e015d70a1a4fe39e0be28a6a906e50d905eff2322164657c8bc906" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:22 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "c039392952a40183fc316beced57190497e643a19c9379685c28827685e416c7" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:25 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "5b4e7ae02354fe6baf38933b5d8e13c4a861da780ed1dd9c086cab531b6b4494" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:28 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1ee6a5406b55b1dda0e4d732abf8c90a22d37d0c38c575aa1b94551d55cebc0b" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:29 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "80fd39abeefa46b82bcb93ba89a31409527be85a481187ff9493157a5bcad94a" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:32 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1057608a4ec10578044fa59de587ec07256d3a5c1d595d2909c3634dc069fedb" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.458: INFO: At 2023-01-22 02:29:34 +0000 UTC - event for pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e: {kubelet ip-172-20-39-135.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "367f36c5ed824f8042a05a0d48faa91d00e62061cad24a7dbc9568063817eecb" network for pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e": networkPlugin cni failed to set up pod "pod-92bfa24a-16a7-4b5c-8a9f-bb42dcf8cf8e_volumemode-1892" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:34:16.608: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:34:16.608: INFO: Jan 22 02:34:16.759: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:16.910: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 11676 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:29:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-22 02:29:19 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:29:59 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-019e723d932658f17 kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-019e723d932658f17,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0f46f4d43a10a69d4,DevicePath:,},},Config:nil,},} Jan 22 02:34:16.910: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:17.062: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:17.371: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:33:44 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:17.372: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:34:17.372: INFO: pod-80862d15-4984-4b9d-99fd-e96e46a3ac67 started at 2023-01-22 02:29:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:17.372: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:17.372: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:17.372: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:17.372: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:17.372: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:30:13 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:17.372: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:34:17.372: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: inline-volume-tester-rkzx7 started at 2023-01-22 02:29:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:17.372: INFO: test-rolling-update-deployment-8656fc4b57-c8wd7 started at 2023-01-22 02:30:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: test-rolling-update-controller-xhnb5 started at 2023-01-22 02:29:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container httpd ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: netserver-0 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:17.372: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:17.372: INFO: Container coredns ready: true, restart count 0 Jan 22 02:34:17.895: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:34:17.895: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:18.045: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 13224 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4110":"csi-mock-csi-mock-volumes-4110","ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:29:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:29:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:34:10 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0603be944c93a7cd3,DevicePath:,},},Config:nil,},} Jan 22 02:34:18.046: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:18.197: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:18.445: INFO: csi-mockplugin-0 started at 2023-01-22 02:27:21 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:18.446: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:18.446: INFO: Container driver-registrar ready: true, restart count 0 Jan 22 02:34:18.446: INFO: Container mock ready: true, restart count 0 Jan 22 02:34:18.446: INFO: pod-projected-configmaps-12ebdd9f-aa1b-4d56-a05e-a8e067698121 started at 2023-01-22 02:29:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container projected-configmap-volume-test ready: false, restart count 0 Jan 22 02:34:18.446: INFO: csi-mockplugin-attacher-0 started at 2023-01-22 02:29:11 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:18.446: INFO: inline-volume-tester-7p6pc started at 2023-01-22 02:30:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:18.446: INFO: service-headless-xgkds started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:34:18.446: INFO: csi-mockplugin-0 started at 2023-01-22 02:29:11 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:18.446: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:34:18.446: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:34:18.446: INFO: Container mock ready: false, restart count 0 Jan 22 02:34:18.446: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-58nkp started at 2023-01-22 02:33:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:18.446: INFO: test-rolling-update-with-lb-7dcdf997cd-2nzq9 started at 2023-01-22 02:30:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container agnhost ready: false, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: pod-configmaps-487886f1-48d2-404e-b45d-500ec77edc3f started at 2023-01-22 02:33:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container env-test ready: false, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:18.446: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:18.446: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:18.446: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:18.446: INFO: test-rolling-update-with-lb-79bf9f6988-rqqm8 started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: dns-test-7165a663-4d55-4543-acfd-a8c273537323 started at 2023-01-22 02:27:52 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:18.446: INFO: Container jessie-querier ready: false, restart count 0 Jan 22 02:34:18.446: INFO: Container querier ready: false, restart count 0 Jan 22 02:34:18.446: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:18.446: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: pvc-volume-tester-plxcq started at 2023-01-22 02:29:54 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container volume-tester ready: false, restart count 0 Jan 22 02:34:18.446: INFO: pod-subpath-test-preprovisionedpv-8dnv started at 2023-01-22 02:33:59 +0000 UTC (0+2 container statuses recorded) Jan 22 02:34:18.446: INFO: Container test-container-subpath-preprovisionedpv-8dnv ready: false, restart count 0 Jan 22 02:34:18.446: INFO: Container test-container-volume-preprovisionedpv-8dnv ready: false, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.446: INFO: metadata-volume-fd30831d-8c17-454a-91a4-d45d8348548b started at 2023-01-22 02:30:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container client-container ready: false, restart count 0 Jan 22 02:34:18.446: INFO: pod-14b5f8c7-e30a-426e-8605-713988dc0cae started at 2023-01-22 02:34:03 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:18.446: INFO: netserver-1 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:18.446: INFO: hostexec-ip-172-20-39-135.ap-northeast-1.compute.internal-2tv2s started at 2023-01-22 02:33:55 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:18.446: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:18.446: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:18.996: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:34:18.996: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:19.147: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 12864 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8079":"ip-172-20-47-230.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:27:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} }]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:33:34 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:19.147: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:19.299: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:19.458: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container coredns ready: true, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.458: INFO: inline-volume-tester2-zmfh6 started at 2023-01-22 02:29:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:34:19.458: INFO: netserver-2 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:19.458: INFO: inline-volume-tester-6nngz started at 2023-01-22 02:27:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.458: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:19.458: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:19.458: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:19.458: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:19.458: INFO: service-headless-p8n6b started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container service-headless ready: false, restart count 0 Jan 22 02:34:19.458: INFO: pod-5e2f43d2-6145-4e2d-8712-db9b840587e7 started at 2023-01-22 02:30:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:34:19.458: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:34:19.458: INFO: test-rolling-update-with-lb-79bf9f6988-44b8c started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.458: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.458: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.459: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.459: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.459: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.459: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:19.459: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:19.459: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:27:08 +0000 UTC (0+7 container statuses recorded) Jan 22 02:34:19.459: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:19.459: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:19.459: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.459: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:19.459: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-dsw6d started at 2023-01-22 02:30:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:19.459: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:34:20.225: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:34:20.225: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:20.375: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 13046 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:28:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:33:48 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:20.375: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:20.527: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:20.684: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:20.684: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:20.684: INFO: service-headless-w4f7z started at 2023-01-22 02:32:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container service-headless ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: pod-906bd654-ba45-4cea-8ad3-0065821bb928 started at 2023-01-22 02:33:49 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container test-container ready: false, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: test-rolling-update-with-lb-79bf9f6988-4fp4m started at 2023-01-22 02:26:59 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container agnhost ready: true, restart count 0 Jan 22 02:34:20.684: INFO: pod-configmaps-220ddd82-1429-45eb-b1eb-7f5627bdf54e started at 2023-01-22 02:29:58 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:34:20.684: INFO: busybox-5e27475f-0f54-40db-a2ce-a1c9dee0380c started at 2023-01-22 02:32:42 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container busybox ready: true, restart count 0 Jan 22 02:34:20.684: INFO: netserver-3 started at 2023-01-22 02:34:09 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container webserver ready: false, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:20.684: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:34:20.684: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:20.684: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:20.684: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:20.684: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:21.170: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:34:21.170: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:21.320: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 12105 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:31:00 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:34:21.321: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:21.472: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:21.629: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:34:21.629: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:34:21.629: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:34:21.629: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:34:21.629: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:34:21.629: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:34:21.629: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:34:21.629: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:34:21.629: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:34:21.629: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:34:21.629: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:34:21.629: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:34:21.629: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:34:21.629: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:34:21.629: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:34:21.629: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:34:22.116: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:34:22.116: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "volumemode-1892" for this suite.
Filter through log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sGeneric\sEphemeral\-volume\s\(default\sfs\)\s\(immediate\-binding\)\]\sephemeral\sshould\ssupport\stwo\spods\swhich\shave\sthe\ssame\svolume\sdefinition$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:214 Jan 22 02:44:48.411: waiting for pod with inline volume Unexpected error: <*errors.errorString | 0xc00025c240>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:349from junit_04.xml
[BeforeEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:29:45.889: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename ephemeral �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should support two pods which have the same volume definition /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:214 Jan 22 02:29:47.220: INFO: Pod inline-volume-79r6x has the following logs: Jan 22 02:29:47.368: INFO: Deleting pod "inline-volume-79r6x" in namespace "ephemeral-1070" Jan 22 02:29:47.517: INFO: Wait up to 5m0s for pod "inline-volume-79r6x" to be fully deleted Jan 22 02:29:47.664: INFO: Creating resource for dynamic PV Jan 22 02:29:47.665: INFO: Using claimSize:1Gi, test suite supported size:{ }, driver(ebs.csi.aws.com) supported size:{ } �[1mSTEP�[0m: creating a StorageClass ephemeral-1070-e2e-sc4tx6d �[1mSTEP�[0m: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil} Jan 22 02:44:48.411: FAIL: waiting for pod with inline volume Unexpected error: <*errors.errorString | 0xc00025c240>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.EphemeralTest.TestEphemeral({{0x7938928, 0xc004060c00}, 0xc001899600, {0xc0044775e0, 0xe}, {0x0, 0x0}, 0xc002f06000, {{0x0, 0x0}, ...}, ...}) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:349 +0x605 k8s.io/kubernetes/test/e2e/storage/testsuites.(*ephemeralTestSuite).DefineTests.func5() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:249 +0x205 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x0?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc000ddc820, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f Jan 22 02:44:48.572: INFO: Error getting logs for pod inline-volume-tester-rkzx7: the server rejected our request for an unknown reason (get pods inline-volume-tester-rkzx7) Jan 22 02:44:48.868: INFO: Deleting pod "inline-volume-tester-rkzx7" in namespace "ephemeral-1070" Jan 22 02:44:49.020: INFO: Wait up to 5m0s for pod "inline-volume-tester-rkzx7" to be fully deleted Jan 22 02:44:51.317: INFO: Wait up to 5m0s for pod PV pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 to be fully deleted Jan 22 02:44:51.317: INFO: Waiting up to 5m0s for PersistentVolume pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 to get deleted Jan 22 02:44:51.465: INFO: PersistentVolume pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 found and phase=Released (148.253645ms) Jan 22 02:44:56.616: INFO: PersistentVolume pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 found and phase=Released (5.298906238s) Jan 22 02:45:01.764: INFO: PersistentVolume pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 found and phase=Released (10.44781732s) Jan 22 02:45:06.916: INFO: PersistentVolume pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 was removed �[1mSTEP�[0m: Deleting sc [AfterEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "ephemeral-1070". �[1mSTEP�[0m: Found 20 events. Jan 22 02:45:07.222: INFO: At 2023-01-22 02:29:46 +0000 UTC - event for inline-volume-79r6x: {default-scheduler } FailedScheduling: 0/5 nodes are available: 5 waiting for ephemeral volume controller to create the persistentvolumeclaim "inline-volume-79r6x-my-volume". Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:47 +0000 UTC - event for inline-volume-79r6x-my-volume: {persistentvolume-controller } ProvisioningFailed: storageclass.storage.k8s.io "no-such-storage-class" not found Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:47 +0000 UTC - event for inline-volume-tester-rkzx7: {default-scheduler } FailedScheduling: 0/5 nodes are available: 5 waiting for ephemeral volume controller to create the persistentvolumeclaim "inline-volume-tester-rkzx7-my-volume-0". Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:47 +0000 UTC - event for inline-volume-tester-rkzx7-my-volume-0: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:47 +0000 UTC - event for inline-volume-tester-rkzx7-my-volume-0: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } Provisioning: External provisioner is provisioning volume for claim "ephemeral-1070/inline-volume-tester-rkzx7-my-volume-0" Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:49 +0000 UTC - event for inline-volume-tester-rkzx7: {default-scheduler } FailedScheduling: 0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims. Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:51 +0000 UTC - event for inline-volume-tester-rkzx7: {default-scheduler } Scheduled: Successfully assigned ephemeral-1070/inline-volume-tester-rkzx7 to ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:51 +0000 UTC - event for inline-volume-tester-rkzx7-my-volume-0: {ebs.csi.aws.com_ip-172-20-58-228_0bd5216f-8c0a-41f6-93fd-52a619f8700c } ProvisioningSucceeded: Successfully provisioned volume pvc-21bb04e2-045b-4612-bee0-b0b7c2565747 Jan 22 02:45:07.223: INFO: At 2023-01-22 02:29:53 +0000 UTC - event for inline-volume-tester-rkzx7: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-21bb04e2-045b-4612-bee0-b0b7c2565747" Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:01 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:01 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "819a73f7a9a9c9022a6d7fde33875509430b653b41e9045ed147ad56145c0e56" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:03 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "a6255dd6670e6446e076a1e3fc2a5fb009d23aa60ae5fc277d8b46e9675a303e" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:05 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "c58ce69b9dcb64d8307012bbff0ff01e5a69a92ed39f98dbf8374e6ea6d597b4" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:05 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "012e6d4a2443d1c38537fe407fbccc0b869c9f6347b14066c63f21570ddb5f42" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:07 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "40fad8f9841d1d69cec06a1c255dea07700360f58ad28886137caf61423ab239" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:08 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "de851c55cdfa04845b18093b911cbaf483e6fe650b0cfe0d0f2f6f9f1ef23e8a" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:09 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f15f366baf5b2e77527edd5f5cd8a078b0c71bf816265ed1a97b75da0fa36415" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:10 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "de451d558d5d2c5f534fa28b2eeba0a4e4640b4fc8d9d4aff2ddda4e7a22d707" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:11 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "31476d04046ab5cc442194b1124fd82f297cffdd02edefb0088a21280307fd4a" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.223: INFO: At 2023-01-22 02:30:12 +0000 UTC - event for inline-volume-tester-rkzx7: {kubelet ip-172-20-33-190.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9bbeee2aafeb8f85a720b5d0ed27c6c72c2fac8a8e0f90e8c9da82e68d6b4044" network for pod "inline-volume-tester-rkzx7": networkPlugin cni failed to set up pod "inline-volume-tester-rkzx7_ephemeral-1070" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:07.371: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:45:07.371: INFO: Jan 22 02:45:07.520: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:07.669: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 18484 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:44:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:45:07.669: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:07.820: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: netserver-0 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-pwlp4 started at 2023-01-22 02:41:33 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:07.977: INFO: rs-tcq62 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container donothing ready: false, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-kh792 started at 2023-01-22 02:40:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container coredns ready: true, restart count 0 Jan 22 02:45:07.977: INFO: pod-subpath-test-preprovisionedpv-89x4 started at 2023-01-22 02:41:44 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Init container init-volume-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:07.977: INFO: Init container test-init-volume-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:07.977: INFO: Container test-container-subpath-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: pod-5f304ab5-0d6e-44a5-aef0-099d371557a8 started at 2023-01-22 02:40:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:07.977: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:07.977: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:07.977: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:07.977: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:07.977: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:07.977: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:07.977: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:07.977: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:08.530: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:08.531: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:08.679: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 17813 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:29:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:29:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:42:12 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-02f9a67e9fa46c78f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02f9a67e9fa46c78f,DevicePath:,},},Config:nil,},} Jan 22 02:45:08.680: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:08.830: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:08.991: INFO: indexed-job-0-qzndn started at 2023-01-22 02:41:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.991: INFO: Container c ready: false, restart count 0 Jan 22 02:45:08.991: INFO: csi-mockplugin-0 started at 2023-01-22 02:42:35 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:08.991: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:45:08.991: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:45:08.991: INFO: Container mock ready: false, restart count 0 Jan 22 02:45:08.991: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.991: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.991: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.991: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.991: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.991: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.991: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:08.991: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:08.991: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:08.991: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:08.991: INFO: rs-sp685 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.991: INFO: Container donothing ready: true, restart count 0 Jan 22 02:45:08.991: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.991: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.991: INFO: csi-mockplugin-resizer-0 started at 2023-01-22 02:42:35 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:45:08.992: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:08.992: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:08.992: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.992: INFO: indexed-job-1-dcvvg started at 2023-01-22 02:41:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container c ready: false, restart count 0 Jan 22 02:45:08.992: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.992: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.992: INFO: indexed-job-2-2cddx started at 2023-01-22 02:42:12 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container c ready: false, restart count 0 Jan 22 02:45:08.992: INFO: netserver-1 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:08.992: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:10 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:08.992: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:45:08.992: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:45:08.992: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:45:08.992: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:45:08.992: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:45:08.992: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:45:08.992: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:45:08.992: INFO: pod-c1563870-772d-4914-a1a9-6ae1736b51eb started at 2023-01-22 02:37:06 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:08.992: INFO: inline-volume-tester-9rmrs started at 2023-01-22 02:41:10 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:45:08.992: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.992: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:08.992: INFO: hostpathsymlink-client started at 2023-01-22 02:40:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container hostpathsymlink-client ready: false, restart count 0 Jan 22 02:45:08.992: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:08.992: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.489: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:09.489: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:09.639: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 17716 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:41:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:41:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0371bc02787c59471],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0371bc02787c59471,DevicePath:,},},Config:nil,},} Jan 22 02:45:09.640: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:09.791: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:09.953: INFO: netserver-2 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: pod-subpath-test-preprovisionedpv-5xtv started at 2023-01-22 02:41:59 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Init container init-volume-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:09.953: INFO: Init container test-init-volume-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:09.953: INFO: Container test-container-subpath-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:09.953: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:09.953: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:09.953: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:09.953: INFO: pod-subpath-test-inlinevolume-zhsl started at 2023-01-22 02:41:49 +0000 UTC (1+2 container statuses recorded) Jan 22 02:45:09.953: INFO: Init container test-init-subpath-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:09.953: INFO: Container test-container-subpath-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:09.953: INFO: Container test-container-volume-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:09.953: INFO: pod-72c4e40c-da11-4c01-bd78-334fabe5b47e started at 2023-01-22 02:41:46 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:45:09.953: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:09.953: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:09.953: INFO: test-rs-58mcf started at 2023-01-22 02:41:33 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container httpd ready: false, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-r4jz4 started at 2023-01-22 02:41:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:09.953: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container coredns ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:09.953: INFO: echo-sourceip started at 2023-01-22 02:41:28 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:09.953: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:45:10.723: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:10.723: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:10.872: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 17489 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5291":"ip-172-20-54-133.ap-northeast-1.compute.internal","csi-hostpath-provisioning-6245":"ip-172-20-54-133.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:38:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:38:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5291^35f2d965-99fe-11ed-a40a-3e2b90db047f kubernetes.io/csi/csi-hostpath-provisioning-6245^47c4b974-99fe-11ed-8f9c-42b9bcafcd1e kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5291^35f2d965-99fe-11ed-a40a-3e2b90db047f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6245^47c4b974-99fe-11ed-8f9c-42b9bcafcd1e,DevicePath:,},},Config:nil,},} Jan 22 02:45:10.872: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:11.023: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:11.222: INFO: pod-subpath-test-inlinevolume-8c5p started at 2023-01-22 02:41:00 +0000 UTC (2+2 container statuses recorded) Jan 22 02:45:11.222: INFO: Init container init-volume-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:11.222: INFO: Init container test-init-subpath-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:11.222: INFO: Container test-container-subpath-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:11.222: INFO: Container test-container-volume-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:11.222: INFO: netserver-3 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container webserver ready: false, restart count 0 Jan 22 02:45:11.222: INFO: sample-webhook-deployment-6c69dbd86b-9ptj7 started at 2023-01-22 02:40:48 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container sample-webhook ready: false, restart count 0 Jan 22 02:45:11.222: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.222: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.222: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:11.222: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:11.222: INFO: indexed-job-3-p9nwq started at 2023-01-22 02:42:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container c ready: false, restart count 0 Jan 22 02:45:11.222: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:23 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:11.222: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:11.222: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:11.222: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:11.222: INFO: pod-subpath-test-inlinevolume-rts6 started at 2023-01-22 02:40:49 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Init container init-volume-inlinevolume-rts6 ready: false, restart count 0 Jan 22 02:45:11.222: INFO: Container test-container-subpath-inlinevolume-rts6 ready: false, restart count 0 Jan 22 02:45:11.222: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.222: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.222: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.222: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.222: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.223: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.223: INFO: pod-subpath-test-inlinevolume-lm48 started at 2023-01-22 02:41:00 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Init container init-volume-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:11.223: INFO: Init container test-init-volume-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:11.223: INFO: Container test-container-subpath-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:11.223: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.223: INFO: pod-subpath-test-inlinevolume-775p started at 2023-01-22 02:41:24 +0000 UTC (1+2 container statuses recorded) Jan 22 02:45:11.223: INFO: Init container test-init-subpath-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:11.223: INFO: Container test-container-subpath-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:11.223: INFO: Container test-container-volume-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:11.223: INFO: hostpath-injector started at 2023-01-22 02:41:32 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container hostpath-injector ready: false, restart count 0 Jan 22 02:45:11.223: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.223: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:40:50 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:11.223: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:11.223: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:11.223: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:11.223: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:45:11.223: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:45:11.223: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:11.223: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:11.223: INFO: pod-subpath-test-dynamicpv-5kk9 started at 2023-01-22 02:41:04 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Init container init-volume-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:11.223: INFO: Init container test-init-subpath-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:11.223: INFO: Container test-container-subpath-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:11.223: INFO: image-pull-test09b56348-f546-4734-be6b-566fb99d2d3e started at 2023-01-22 02:41:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container image-pull-test ready: false, restart count 0 Jan 22 02:45:11.223: INFO: rs-bz52n started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container donothing ready: false, restart count 0 Jan 22 02:45:11.223: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:11.223: INFO: pod-efed6c88-76fd-4889-8a42-f736d44b87a6 started at 2023-01-22 02:41:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:11.223: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:11.937: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:11.937: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:12.086: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 16931 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-09cb3f0e26fe23ead,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3918811136 0} {<nil>} 3826964Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3813953536 0} {<nil>} 3724564Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:18:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:11 +0000 UTC,LastTransitionTime:2023-01-22 02:19:36 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.228,},NodeAddress{Type:ExternalIP,Address:54.95.21.195,},NodeAddress{Type:Hostname,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-58-228.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-21-195.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec27b0d3a396382ff5f0a0270a9482a1,SystemUUID:ec27b0d3-a396-382f-f5f0-a0270a9482a1,BootID:cb97f4db-a02f-4efb-90b8-c0f2ca827d55,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:662209237,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.23.16 registry.k8s.io/kube-apiserver-amd64:v1.23.16],SizeBytes:129999849,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.23.16 registry.k8s.io/kube-controller-manager-amd64:v1.23.16],SizeBytes:119940367,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:110016956,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:105637719,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:102695546,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57736070,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:55464778,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:54848595,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.23.16 registry.k8s.io/kube-scheduler-amd64:v1.23.16],SizeBytes:51852546,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:8786911,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:45:12.087: INFO: Logging kubelet events for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:12.237: INFO: Logging pods the kubelet thinks is on node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:12.392: INFO: kube-apiserver-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+2 container statuses recorded) Jan 22 02:45:12.392: INFO: Container healthcheck ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container kube-apiserver ready: true, restart count 1 Jan 22 02:45:12.392: INFO: kube-controller-manager-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.392: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 22 02:45:12.392: INFO: etcd-manager-events-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.392: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:45:12.392: INFO: ebs-csi-node-zbtcm started at 2023-01-22 02:19:12 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:12.392: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:12.392: INFO: ebs-csi-controller-6d664d6f54-hgmsx started at 2023-01-22 02:19:15 +0000 UTC (0+5 container statuses recorded) Jan 22 02:45:12.392: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:12.392: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:12.392: INFO: kops-controller-zsnxq started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.392: INFO: Container kops-controller ready: true, restart count 0 Jan 22 02:45:12.392: INFO: kube-scheduler-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.393: INFO: Container kube-scheduler ready: true, restart count 0 Jan 22 02:45:12.393: INFO: etcd-manager-main-ip-172-20-58-228.ap-northeast-1.compute.internal started at 2023-01-22 02:18:08 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.393: INFO: Container etcd-manager ready: true, restart count 0 Jan 22 02:45:12.393: INFO: cilium-7rvrk started at 2023-01-22 02:19:12 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:12.393: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:12.393: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:12.393: INFO: cilium-operator-d47487c76-s6k52 started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.393: INFO: Container cilium-operator ready: true, restart count 0 Jan 22 02:45:12.393: INFO: dns-controller-7db8b74b99-7cz9r started at 2023-01-22 02:19:15 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:12.393: INFO: Container dns-controller ready: true, restart count 0 Jan 22 02:45:12.866: INFO: Latency metrics for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:12.866: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "ephemeral-1070" for this suite.
Find with mentions in log files
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sbe\sable\sto\sdeny\spod\sand\sconfigmap\screation\s\[Conformance\]$'
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/webhook.go:87 Jan 22 02:45:48.834: waiting for the deployment status valid%!(EXTRA string=k8s.gcr.io/e2e-test-images/agnhost:2.39, string=sample-webhook-deployment, string=webhook-1265) Unexpected error: <*errors.errorString | 0xc002dacde0>: { s: "error waiting for deployment \"sample-webhook-deployment\" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:\"Available\", Status:\"False\", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:\"MinimumReplicasUnavailable\", Message:\"Deployment does not have minimum availability.\"}, v1.DeploymentCondition{Type:\"Progressing\", Status:\"True\", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:\"ReplicaSetUpdated\", Message:\"ReplicaSet \\\"sample-webhook-deployment-6c69dbd86b\\\" is progressing.\"}}, CollisionCount:(*int32)(nil)}", } error waiting for deployment "sample-webhook-deployment" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/webhook.go:824
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185 �[1mSTEP�[0m: Creating a kubernetes client Jan 22 02:40:45.845: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename webhook �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/webhook.go:87 �[1mSTEP�[0m: Setting up server cert �[1mSTEP�[0m: Create role binding to let webhook read extension-apiserver-authentication �[1mSTEP�[0m: Deploying the webhook pod �[1mSTEP�[0m: Wait for the deployment to be ready Jan 22 02:40:48.537: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:40:50.691: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:40:52.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:40:54.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:40:56.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:40:58.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:00.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:02.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:04.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:06.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:08.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:10.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:12.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:14.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:16.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:18.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:20.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:22.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:24.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:26.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:28.687: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:30.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:32.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:34.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:36.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:38.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:40.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:42.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:44.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:46.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:48.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:50.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:52.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:54.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:56.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:41:58.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:00.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:02.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:04.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:06.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:08.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:10.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:12.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:14.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:16.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:18.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:20.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:22.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:24.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:26.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:28.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:30.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:32.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:34.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:36.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:38.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:40.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:42.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:44.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:46.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:48.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:50.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:52.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:54.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:56.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:42:58.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:00.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:02.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:04.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:06.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:08.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:10.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:12.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:14.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:16.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:18.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:20.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:22.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:24.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:26.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:28.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:30.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:32.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:34.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:36.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:38.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:40.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:42.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:44.687: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:46.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:48.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:50.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:52.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:54.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:56.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:43:58.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:00.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:02.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:04.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:06.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:08.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:10.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:12.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:14.696: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:16.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:18.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:20.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:22.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:24.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:26.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:28.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:30.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:32.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:34.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:36.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:38.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:40.690: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:42.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:44.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:46.694: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:48.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:50.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:52.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:54.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:56.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:44:58.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:00.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:02.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:04.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:06.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:08.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:10.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:12.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:14.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:16.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:18.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:20.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:22.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:24.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:26.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:28.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:30.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:32.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:34.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:36.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:38.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:40.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:42.691: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:44.685: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:46.686: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:48.684: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:48.832: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} Jan 22 02:45:48.833: FAIL: waiting for the deployment status valid%!(EXTRA string=k8s.gcr.io/e2e-test-images/agnhost:2.39, string=sample-webhook-deployment, string=webhook-1265) Unexpected error: <*errors.errorString | 0xc002dacde0>: { s: "error waiting for deployment \"sample-webhook-deployment\" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:\"Available\", Status:\"False\", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:\"MinimumReplicasUnavailable\", Message:\"Deployment does not have minimum availability.\"}, v1.DeploymentCondition{Type:\"Progressing\", Status:\"True\", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:\"ReplicaSetUpdated\", Message:\"ReplicaSet \\\"sample-webhook-deployment-6c69dbd86b\\\" is progressing.\"}}, CollisionCount:(*int32)(nil)}", } error waiting for deployment "sample-webhook-deployment" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), LastTransitionTime:time.Date(2023, time.January, 22, 2, 40, 48, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-6c69dbd86b\" is progressing."}}, CollisionCount:(*int32)(nil)} occurred Full Stack Trace k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc00093d8c0, {0xc003468750, 0x27}, 0xc00578ed70, 0x20fb, 0x20fc) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/webhook.go:824 +0xeb0 k8s.io/kubernetes/test/e2e/apimachinery.glob..func27.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apimachinery/webhook.go:99 +0x226 k8s.io/kubernetes/test/e2e.RunE2ETests(0x24eec17?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:133 +0x6b7 k8s.io/kubernetes/test/e2e.TestE2E(0x24602d9?) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00073dba0, 0x72ecb90) /usr/local/go/src/testing/testing.go:1446 +0x10b created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1493 +0x35f [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Collecting events from namespace "webhook-1265". �[1mSTEP�[0m: Found 14 events. Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:48 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-6c69dbd86b to 1 Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:48 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-6c69dbd86b-9ptj7 Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:48 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {default-scheduler } Scheduled: Successfully assigned webhook-1265/sample-webhook-deployment-6c69dbd86b-9ptj7 to ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:49 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "cf1b982af7993ea73db98032600dda1850943a59ca68327e39f5ce6ea5c218a8" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:49 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:52 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "02f50a46b3b663e9fb62e46ac21a0ffc4880158a7840af967092f1182d9948f2" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:55 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f362013d594bb41f353b65c400c34e9dd7adeb7c80b0770930ebc90f34b1167c" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:56 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "369ee8b7982d1e57c84dfbd061b3da7e31a1d6df1a6fd72d4e332836d7898047" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:40:58 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "a45ddf8dd08e1208fdd68e6972fb80cc2f83f6187cc7c2aa0741a6b096b08474" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:41:01 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "6ed98982cef71d223ba666158a5c6b5fd0e34ecdf298e0707d8207b7db66fcf5" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:41:04 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "4dcd01c2c11a9cc988c042ba6729042df1ccd92c81b0205efe76ff15cb1afb77" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:41:08 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "e94813cdc214d9a42a8a7501801ef8da2caad8d93c41d5cd8166c4bb4a5e792a" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:41:11 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1c3fcfcd4dcc3bc30fe996207c8d56cf770f0cc4db91ff810bfa1b1ea62587ea" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:48.982: INFO: At 2023-01-22 02:41:14 +0000 UTC - event for sample-webhook-deployment-6c69dbd86b-9ptj7: {kubelet ip-172-20-54-133.ap-northeast-1.compute.internal} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "a51637d63062a04bfb92e296f1669b6bebec4b9f5231dfc9be848c05d9bf9031" network for pod "sample-webhook-deployment-6c69dbd86b-9ptj7": networkPlugin cni failed to set up pod "sample-webhook-deployment-6c69dbd86b-9ptj7_webhook-1265" network: unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 22 02:45:49.130: INFO: POD NODE PHASE GRACE CONDITIONS Jan 22 02:45:49.130: INFO: sample-webhook-deployment-6c69dbd86b-9ptj7 ip-172-20-54-133.ap-northeast-1.compute.internal Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:40:48 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:40:48 +0000 UTC ContainersNotReady containers with unready status: [sample-webhook]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:40:48 +0000 UTC ContainersNotReady containers with unready status: [sample-webhook]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-22 02:40:48 +0000 UTC }] Jan 22 02:45:49.130: INFO: Jan 22 02:45:49.288: INFO: Unable to fetch webhook-1265/sample-webhook-deployment-6c69dbd86b-9ptj7/sample-webhook logs: the server rejected our request for an unknown reason (get pods sample-webhook-deployment-6c69dbd86b-9ptj7) Jan 22 02:45:49.437: INFO: Logging node info for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:49.585: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-33-190.ap-northeast-1.compute.internal 4ad0b1c2-7558-498a-942b-d13c8f786187 18484 0 2023-01-22 02:20:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-33-190.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-33-190.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0749ba46aef53107a"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:44:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0749ba46aef53107a,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:44:51 +0000 UTC,LastTransitionTime:2023-01-22 02:20:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.33.190,},NodeAddress{Type:ExternalIP,Address:43.206.227.161,},NodeAddress{Type:Hostname,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-33-190.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-206-227-161.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec24547d270b9e634891a288c569a232,SystemUUID:ec24547d-270b-9e63-4891-a288c569a232,BootID:75d70e17-4682-434a-bb0c-a7ab4ca457ff,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf k8s.gcr.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:44576952,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:45:49.585: INFO: Logging kubelet events for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:49.735: INFO: Logging pods the kubelet thinks is on node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:49.898: INFO: ebs-csi-node-cwkdj started at 2023-01-22 02:20:37 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:49.898: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:49.898: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:49.898: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:49.898: INFO: pod-5f304ab5-0d6e-44a5-aef0-099d371557a8 started at 2023-01-22 02:40:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6cjs5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cilium-zmtpz started at 2023-01-22 02:20:37 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:49.898: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-54gfw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: netserver-0 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:49.898: INFO: rs-tcq62 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container donothing ready: false, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-6vdq7 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-9l2r9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-pwlp4 started at 2023-01-22 02:41:33 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqjp9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: hostexec-ip-172-20-33-190.ap-northeast-1.compute.internal-kh792 started at 2023-01-22 02:40:41 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-bpdmb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-dqddf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-hrf8q started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: pod-subpath-test-preprovisionedpv-89x4 started at 2023-01-22 02:41:44 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Init container init-volume-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:49.898: INFO: Init container test-init-volume-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:49.898: INFO: Container test-container-subpath-preprovisionedpv-89x4 ready: false, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-k99tm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:49.898: INFO: coredns-867df8f45c-m72pb started at 2023-01-22 02:21:16 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container coredns ready: true, restart count 0 Jan 22 02:45:49.898: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-l76tj started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:49.898: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.372: INFO: Latency metrics for node ip-172-20-33-190.ap-northeast-1.compute.internal Jan 22 02:45:50.373: INFO: Logging node info for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:50.520: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-39-135.ap-northeast-1.compute.internal f4486cc6-5ab1-43a4-863e-d2d0f17ab0a5 18675 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-39-135.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bdff716c5b98059c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-22 02:45:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0bdff716c5b98059c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:45:26 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:45:26 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:45:26 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:45:26 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.135,},NodeAddress{Type:ExternalIP,Address:13.115.25.150,},NodeAddress{Type:Hostname,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-39-135.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-115-25-150.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec266a16efcbc3e59cae7c0c76431947,SystemUUID:ec266a16-efcb-c3e5-9cae-7c0c76431947,BootID:0085cdb9-6e0b-4a3d-8b39-0e39532a1fcd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 22 02:45:50.520: INFO: Logging kubelet events for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:50.669: INFO: Logging pods the kubelet thinks is on node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-8pl2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-gzrln started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: hostpathsymlink-client started at 2023-01-22 02:40:51 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container hostpathsymlink-client ready: false, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zqjn9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: indexed-job-0-qzndn started at 2023-01-22 02:41:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container c ready: false, restart count 0 Jan 22 02:45:50.852: INFO: csi-mockplugin-0 started at 2023-01-22 02:42:35 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:50.852: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container driver-registrar ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container mock ready: false, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-n5nt4 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: ebs-csi-node-9ctnr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:50.852: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:50.852: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:50.852: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-d8fjx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-86nbx started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: rs-sp685 started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container donothing ready: true, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fqlpf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: csi-mockplugin-resizer-0 started at 2023-01-22 02:42:35 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:45:50.852: INFO: cilium-9v866 started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:50.852: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-4cb79 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: indexed-job-1-dcvvg started at 2023-01-22 02:41:53 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container c ready: false, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-t8qg5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q9g4f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:50.852: INFO: indexed-job-2-2cddx started at 2023-01-22 02:42:12 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container c ready: false, restart count 0 Jan 22 02:45:50.852: INFO: netserver-1 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:50.852: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:10 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:50.852: INFO: Container csi-attacher ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container csi-provisioner ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container csi-resizer ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container hostpath ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container liveness-probe ready: false, restart count 0 Jan 22 02:45:50.852: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 22 02:45:50.852: INFO: inline-volume-tester-9rmrs started at 2023-01-22 02:41:10 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:50.852: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 22 02:45:51.645: INFO: Latency metrics for node ip-172-20-39-135.ap-northeast-1.compute.internal Jan 22 02:45:51.645: INFO: Logging node info for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:51.793: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-47-230.ap-northeast-1.compute.internal 4c266ecd-98f6-44b7-8cb9-641dc1cf8698 17716 0 2023-01-22 02:20:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-47-230.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-47-230.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d4f1bcc618ac95c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-22 02:20:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-01-22 02:20:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:41:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:41:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-06d4f1bcc618ac95c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:53 +0000 UTC,LastTransitionTime:2023-01-22 02:20:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.47.230,},NodeAddress{Type:ExternalIP,Address:54.95.174.250,},NodeAddress{Type:Hostname,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-47-230.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-95-174-250.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2c3e60643f41abb97fcc6d726a1564,SystemUUID:ec2c3e60-643f-41ab-b97f-cc6d726a1564,BootID:4bb2157a-ddf6-40e0-8268-f28e5d2b53fd,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253371792,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:c8e03f60afa90a28e4bb6ec9a8d0fc36d89de4b7475cf2d613afa793ec969fe0 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.0],SizeBytes:56382329,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:50249337,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:49176472,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:47910609,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:19662887,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[k8s.gcr.io/sig-storage/mock-driver@sha256:a7b517f9e0f42ffade802eef9cefa271372386b85f55b702b493241e58459793 k8s.gcr.io/sig-storage/mock-driver:v4.1.0],SizeBytes:17680993,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0371bc02787c59471],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0371bc02787c59471,DevicePath:,},},Config:nil,},} Jan 22 02:45:51.793: INFO: Logging kubelet events for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:51.944: INFO: Logging pods the kubelet thinks is on node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pvjtg started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pszm4 started at 2023-01-22 02:27:25 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ggbjk started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.101: INFO: cilium-k8gp2 started at 2023-01-22 02:20:27 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:52.101: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:52.101: INFO: test-rs-58mcf started at 2023-01-22 02:41:33 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container httpd ready: false, restart count 0 Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-b7jqb started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.101: INFO: hostexec-ip-172-20-47-230.ap-northeast-1.compute.internal-r4jz4 started at 2023-01-22 02:41:50 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container agnhost-container ready: true, restart count 0 Jan 22 02:45:52.101: INFO: coredns-867df8f45c-vsp94 started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container coredns ready: true, restart count 0 Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-s2qc6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2rtjm started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.101: INFO: echo-sourceip started at 2023-01-22 02:41:28 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container agnhost-container ready: false, restart count 0 Jan 22 02:45:52.101: INFO: netserver-2 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container webserver ready: true, restart count 0 Jan 22 02:45:52.101: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-66gvr started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.101: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.102: INFO: pod-subpath-test-preprovisionedpv-5xtv started at 2023-01-22 02:41:59 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:52.102: INFO: Init container init-volume-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:52.102: INFO: Init container test-init-volume-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:52.102: INFO: Container test-container-subpath-preprovisionedpv-5xtv ready: false, restart count 0 Jan 22 02:45:52.102: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-sr4xh started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.102: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.102: INFO: ebs-csi-node-f2g4d started at 2023-01-22 02:20:27 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:52.102: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:52.102: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:52.102: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:52.102: INFO: pod-subpath-test-inlinevolume-zhsl started at 2023-01-22 02:41:49 +0000 UTC (1+2 container statuses recorded) Jan 22 02:45:52.102: INFO: Init container test-init-subpath-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:52.102: INFO: Container test-container-subpath-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:52.102: INFO: Container test-container-volume-inlinevolume-zhsl ready: false, restart count 0 Jan 22 02:45:52.102: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pcknd started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.102: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: false, restart count 0 Jan 22 02:45:52.102: INFO: pod-72c4e40c-da11-4c01-bd78-334fabe5b47e started at 2023-01-22 02:41:46 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.102: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:52.102: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-q4q2p started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.102: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:52.102: INFO: coredns-autoscaler-557ccb4c66-6zqjf started at 2023-01-22 02:20:47 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:52.102: INFO: Container autoscaler ready: true, restart count 0 Jan 22 02:45:52.592: INFO: Latency metrics for node ip-172-20-47-230.ap-northeast-1.compute.internal Jan 22 02:45:52.592: INFO: Logging node info for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:52.739: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-54-133.ap-northeast-1.compute.internal 576aac5d-2679-4caa-9e44-7353f5aa5ef4 17489 0 2023-01-22 02:20:29 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-54-133.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:node node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.hostpath.csi/node:ip-172-20-54-133.ap-northeast-1.compute.internal topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5291":"ip-172-20-54-133.ap-northeast-1.compute.internal","csi-hostpath-provisioning-6245":"ip-172-20-54-133.ap-northeast-1.compute.internal","ebs.csi.aws.com":"i-0dabedb3716bde746"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kops-controller Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-22 02:20:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {e2e.test Update v1 2023-01-22 02:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:38:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-22 02:38:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-1a/i-0dabedb3716bde746,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{50478186496 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4078190592 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{45430367772 0} {<nil>} 45430367772 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3973332992 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:29 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-22 02:41:39 +0000 UTC,LastTransitionTime:2023-01-22 02:20:50 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.133,},NodeAddress{Type:ExternalIP,Address:35.77.98.5,},NodeAddress{Type:Hostname,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:InternalDNS,Address:ip-172-20-54-133.ap-northeast-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-77-98-5.ap-northeast-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d1298374492a277d8bbec20161c72,SystemUUID:ec2d1298-3744-92a2-77d8-bbec20161c72,BootID:d02b667a-ff32-4b96-a05d-0e0ce3448866,KernelVersion:4.19.0-23-cloud-amd64,OSImage:Debian GNU/Linux 10 (buster),ContainerRuntimeVersion:docker://20.10.17,KubeletVersion:v1.23.16,KubeProxyVersion:v1.23.16,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:456569528,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:20f25f275d46aa728f7615a1ccc19c78b2ed89435bf943a44b339f70f45508e6 k8s.gcr.io/e2e-test-images/httpd:2.4.39-2],SizeBytes:126894770,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:7e8bdd271312fd25fc5ff5a8f04727be84044eb3d7d8d03611972a6752e2e11e k8s.gcr.io/e2e-test-images/agnhost:2.39],SizeBytes:126872991,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/httpd@sha256:1b9d1b2f36cb2dbee1960e82a9344aeb11bd4c4c03abf5e1853e0559c23855e3 k8s.gcr.io/e2e-test-images/httpd:2.4.38-2],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.23.16 registry.k8s.io/kube-proxy-amd64:v1.23.16],SizeBytes:110832791,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:94471559,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:4e74c0492bceddc598de1c90cc5bc14dcda94cb49fa9c5bad9d117c4834b5e08 k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1],SizeBytes:56379269,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:36c31f7e1f433c9634d24f876353e8646246d81a03c4e351202c2644daff1620 k8s.gcr.io/sig-storage/csi-resizer:v1.2.0],SizeBytes:53956308,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:ed98431376c9e944e19a465fe8ea944806714dd95416a0821096c78d66b579bd k8s.gcr.io/sig-storage/csi-snapshotter:v4.1.1],SizeBytes:53531990,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:c5be65d6679efabb969d9b019300d187437ae876f992c40911fd2892bbef3b36 k8s.gcr.io/sig-storage/csi-attacher:v3.2.0],SizeBytes:53500826,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:60ab9b3e6a030d3038c87c0d6bca2930f58d1d72823e6a4af09767dc83b696a2 k8s.gcr.io/sig-storage/csi-attacher:v3.2.1],SizeBytes:53499570,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:232fe80174d60d520d36043103853a1d7ab4b7f3782cf43e45034f04ccda58ce k8s.gcr.io/sig-storage/hostpathplugin:v1.7.1],SizeBytes:29977921,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:19582112,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:2dee3fe5fe861bb66c3a4ac51114f3447a4cd35870e0f2e2b558c7a400d89589 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0],SizeBytes:18699436,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17611980,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:17175488,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16032814,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac k8s.gcr.io/e2e-test-images/nonewprivs:1.3],SizeBytes:7107254,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db k8s.gcr.io/pause:3.6 registry.k8s.io/pause:3.6],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5291^35f2d965-99fe-11ed-a40a-3e2b90db047f kubernetes.io/csi/csi-hostpath-provisioning-6245^47c4b974-99fe-11ed-8f9c-42b9bcafcd1e kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-009c00a8030d4d7c8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5291^35f2d965-99fe-11ed-a40a-3e2b90db047f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6245^47c4b974-99fe-11ed-8f9c-42b9bcafcd1e,DevicePath:,},},Config:nil,},} Jan 22 02:45:52.740: INFO: Logging kubelet events for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:52.890: INFO: Logging pods the kubelet thinks is on node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:53.051: INFO: pod-subpath-test-inlinevolume-lm48 started at 2023-01-22 02:41:00 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Init container init-volume-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:53.051: INFO: Init container test-init-volume-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:53.051: INFO: Container test-container-subpath-inlinevolume-lm48 ready: false, restart count 0 Jan 22 02:45:53.051: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-w85vc started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.051: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-jlv9f started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.051: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-zsh6v started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.051: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-ltbxf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.051: INFO: emptydir-injector started at 2023-01-22 02:45:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container emptydir-injector ready: false, restart count 0 Jan 22 02:45:53.051: INFO: hostpath-injector started at 2023-01-22 02:41:32 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container hostpath-injector ready: false, restart count 0 Jan 22 02:45:53.051: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-7p4c6 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.051: INFO: pod-subpath-test-inlinevolume-775p started at 2023-01-22 02:41:24 +0000 UTC (1+2 container statuses recorded) Jan 22 02:45:53.051: INFO: Init container test-init-subpath-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:53.051: INFO: Container test-container-subpath-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:53.051: INFO: Container test-container-volume-inlinevolume-775p ready: false, restart count 0 Jan 22 02:45:53.051: INFO: pod-subpath-test-dynamicpv-5kk9 started at 2023-01-22 02:41:04 +0000 UTC (2+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Init container init-volume-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:53.051: INFO: Init container test-init-subpath-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:53.051: INFO: Container test-container-subpath-dynamicpv-5kk9 ready: false, restart count 0 Jan 22 02:45:53.051: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-xw5k9 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.051: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.051: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:40:50 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:53.051: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:53.051: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:53.051: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:53.051: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:45:53.051: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:45:53.051: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:53.051: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:53.051: INFO: rs-bz52n started at 2023-01-22 02:39:37 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container donothing ready: false, restart count 0 Jan 22 02:45:53.052: INFO: image-pull-test09b56348-f546-4734-be6b-566fb99d2d3e started at 2023-01-22 02:41:22 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container image-pull-test ready: false, restart count 0 Jan 22 02:45:53.052: INFO: downward-api-b0cb9a3d-ef7f-48bc-a717-294da7b8ee0e started at 2023-01-22 02:45:14 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container dapi-container ready: false, restart count 0 Jan 22 02:45:53.052: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-2n5fw started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.052: INFO: pod-efed6c88-76fd-4889-8a42-f736d44b87a6 started at 2023-01-22 02:41:02 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container write-pod ready: false, restart count 0 Jan 22 02:45:53.052: INFO: pod-subpath-test-inlinevolume-8c5p started at 2023-01-22 02:41:00 +0000 UTC (2+2 container statuses recorded) Jan 22 02:45:53.052: INFO: Init container init-volume-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:53.052: INFO: Init container test-init-subpath-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:53.052: INFO: Container test-container-subpath-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:53.052: INFO: Container test-container-volume-inlinevolume-8c5p ready: false, restart count 0 Jan 22 02:45:53.052: INFO: indexed-job-3-p9nwq started at 2023-01-22 02:42:17 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container c ready: false, restart count 0 Jan 22 02:45:53.052: INFO: netserver-3 started at 2023-01-22 02:40:13 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container webserver ready: false, restart count 0 Jan 22 02:45:53.052: INFO: sample-webhook-deployment-6c69dbd86b-9ptj7 started at 2023-01-22 02:40:48 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container sample-webhook ready: false, restart count 0 Jan 22 02:45:53.052: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-fj2p5 started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.052: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-c4llf started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.052: INFO: ebs-csi-node-28jrr started at 2023-01-22 02:20:30 +0000 UTC (0+3 container statuses recorded) Jan 22 02:45:53.052: INFO: Container ebs-plugin ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:53.052: INFO: pod-subpath-test-inlinevolume-rts6 started at 2023-01-22 02:40:49 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Init container init-volume-inlinevolume-rts6 ready: false, restart count 0 Jan 22 02:45:53.052: INFO: Container test-container-subpath-inlinevolume-rts6 ready: false, restart count 0 Jan 22 02:45:53.052: INFO: csi-hostpathplugin-0 started at 2023-01-22 02:41:23 +0000 UTC (0+7 container statuses recorded) Jan 22 02:45:53.052: INFO: Container csi-attacher ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container csi-provisioner ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container csi-resizer ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container hostpath ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container liveness-probe ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 22 02:45:53.052: INFO: cilium-84ncp started at 2023-01-22 02:20:30 +0000 UTC (1+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 22 02:45:53.052: INFO: Container cilium-agent ready: true, restart count 0 Jan 22 02:45:53.052: INFO: cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407-pqdvl started at 2023-01-22 02:27:24 +0000 UTC (0+1 container statuses recorded) Jan 22 02:45:53.052: INFO: Container cleanup40-45ee51c1-e1fe-4660-bc72-fe12b5460407 ready: true, restart count 0 Jan 22 02:45:53.843: INFO: Latency metrics for node ip-172-20-54-133.ap-northeast-1.compute.internal Jan 22 02:45:53.844: INFO: Logging node info for node ip-172-20-58-228.ap-northeast-1.compute.internal Jan 22 02:45:53.993: INFO: Node Info: &Node{ObjectMeta:{ip-172-20-58-228.ap-northeast-1.compute.internal db4356ca-789d-4cde-b55f-08a19be769f3 16931 0 2023-01-22 02:18:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-1 failure-domain.beta.kubernetes.io/zone:ap-northeast-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-172-20-58-228.ap-northeast-1.compute.internal kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/control-plane: node-role.kubernetes.io/master: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-1a topology.kubernetes.io/region:ap-northeast-1 topology.kubernetes.io/zone:ap-northeast-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cb3f0e26fe23ead"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-22 02:18:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {protokube Update v1 2023-01-22 02:19:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kops-controller Update v1 2023-01-22 02:19:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2023-01-22 02:19:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-22 02:20:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"