e2e-kops-grid-cilium-eni-u2004-k26 #1608346948505440256
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 34 failed / 855 succeeded |
| Started | |
| Elapsed | 40m12s |
| Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
test/e2e/storage/testsuites/volume_expand.go:238 k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() test/e2e/storage/testsuites/volume_expand.go:238 +0xe5afrom junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:33:25.563�[0m Dec 29 06:33:25.563: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename volume-expand �[38;5;243m12/29/22 06:33:25.564�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:33:25.655�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:33:25.712�[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand test/e2e/framework/metrics/init/init.go:31 [It] Verify if offline PVC expansion works test/e2e/storage/testsuites/volume_expand.go:172 Dec 29 06:33:25.769: INFO: Creating resource for dynamic PV Dec 29 06:33:25.769: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} �[1mSTEP:�[0m creating a StorageClass volume-expand-5953-e2e-schqfhx �[38;5;243m12/29/22 06:33:25.77�[0m �[1mSTEP:�[0m creating a claim �[38;5;243m12/29/22 06:33:25.802�[0m Dec 29 06:33:25.802: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP:�[0m Creating a pod with dynamically provisioned volume �[38;5;243m12/29/22 06:33:25.864�[0m Dec 29 06:33:25.895: INFO: Waiting up to 5m0s for pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe" in namespace "volume-expand-5953" to be "running" Dec 29 06:33:25.927: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 31.043118ms Dec 29 06:33:27.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 2.061367686s Dec 29 06:33:29.962: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 4.066515522s Dec 29 06:33:31.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 6.061484065s Dec 29 06:33:33.956: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 8.060661102s Dec 29 06:33:35.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 10.061663339s Dec 29 06:33:37.958: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 12.062264351s Dec 29 06:33:39.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 14.061482889s Dec 29 06:33:41.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 16.061269235s Dec 29 06:33:43.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 18.061395131s Dec 29 06:33:45.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Pending", Reason="", readiness=false. Elapsed: 20.061089534s Dec 29 06:33:47.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe": Phase="Running", Reason="", readiness=true. Elapsed: 22.061258761s Dec 29 06:33:47.957: INFO: Pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe" satisfied condition "running" �[1mSTEP:�[0m Deleting the previously created pod �[38;5;243m12/29/22 06:33:47.986�[0m Dec 29 06:33:47.986: INFO: Deleting pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe" in namespace "volume-expand-5953" Dec 29 06:33:48.022: INFO: Wait up to 5m0s for pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe" to be fully deleted �[1mSTEP:�[0m Expanding current pvc �[38;5;243m12/29/22 06:33:52.083�[0m Dec 29 06:33:52.083: INFO: currentPvcSize {{1073741824 0} {<nil>} 1Gi BinarySI}, newSize {{2147483648 0} {<nil>} BinarySI} �[1mSTEP:�[0m Waiting for cloudprovider resize to finish �[38;5;243m12/29/22 06:33:52.145�[0m �[1mSTEP:�[0m Checking for conditions on pvc �[38;5;243m12/29/22 06:33:56.229�[0m �[1mSTEP:�[0m Creating a new pod with same volume �[38;5;243m12/29/22 06:33:56.258�[0m Dec 29 06:33:56.305: INFO: Waiting up to 10m0s for pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351" in namespace "volume-expand-5953" to be "running" Dec 29 06:33:56.334: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 29.163346ms Dec 29 06:33:58.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2.059189111s Dec 29 06:34:00.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4.059095551s Dec 29 06:34:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6.059414595s Dec 29 06:34:04.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8.059274065s Dec 29 06:34:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 10.059505786s Dec 29 06:34:08.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 12.060663368s Dec 29 06:34:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 14.059905849s Dec 29 06:34:12.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 16.059153358s Dec 29 06:34:14.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 18.059002527s Dec 29 06:34:16.367: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 20.062289277s Dec 29 06:34:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 22.060253416s Dec 29 06:34:20.373: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 24.068279897s Dec 29 06:34:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 26.059396477s Dec 29 06:34:24.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 28.059294012s Dec 29 06:34:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 30.059528635s Dec 29 06:34:28.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 32.059261482s Dec 29 06:34:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 34.059516552s Dec 29 06:34:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 36.059609901s Dec 29 06:34:34.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 38.059875628s Dec 29 06:34:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 40.059739897s Dec 29 06:34:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 42.059788702s Dec 29 06:34:40.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 44.059262524s Dec 29 06:34:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 46.059789484s Dec 29 06:34:44.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 48.059502067s Dec 29 06:34:46.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 50.059337039s Dec 29 06:34:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 52.059491641s Dec 29 06:34:50.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 54.06015102s Dec 29 06:34:52.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 56.06053465s Dec 29 06:34:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 58.059900995s Dec 29 06:34:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.059563074s Dec 29 06:34:58.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.060348651s Dec 29 06:35:00.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.059261682s Dec 29 06:35:02.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.060861807s Dec 29 06:35:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.060189689s Dec 29 06:35:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.059568529s Dec 29 06:35:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.060178575s Dec 29 06:35:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.059902851s Dec 29 06:35:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.05971027s Dec 29 06:35:14.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.060447379s Dec 29 06:35:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.060332816s Dec 29 06:35:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.05955528s Dec 29 06:35:20.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.060111055s Dec 29 06:35:22.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.059324234s Dec 29 06:35:24.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.061321217s Dec 29 06:35:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.05970489s Dec 29 06:35:28.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.059713959s Dec 29 06:35:30.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.060649284s Dec 29 06:35:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.060089582s Dec 29 06:35:34.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.060025367s Dec 29 06:35:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.059484573s Dec 29 06:35:38.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.060408368s Dec 29 06:35:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.059661664s Dec 29 06:35:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.060315042s Dec 29 06:35:44.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.059523555s Dec 29 06:35:46.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.060065762s Dec 29 06:35:48.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.060715458s Dec 29 06:35:50.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.059366371s Dec 29 06:35:52.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.059545089s Dec 29 06:35:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.05979995s Dec 29 06:35:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.060168613s Dec 29 06:35:58.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.060444737s Dec 29 06:36:00.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.059340858s Dec 29 06:36:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.059644342s Dec 29 06:36:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.059918875s Dec 29 06:36:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.059530814s Dec 29 06:36:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.060033453s Dec 29 06:36:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.059447497s Dec 29 06:36:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.060241861s Dec 29 06:36:14.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.060363611s Dec 29 06:36:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.059454246s Dec 29 06:36:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.060181446s Dec 29 06:36:20.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.06086301s Dec 29 06:36:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.059447662s Dec 29 06:36:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.059948091s Dec 29 06:36:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.059621261s Dec 29 06:36:28.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.059155081s Dec 29 06:36:30.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.059313516s Dec 29 06:36:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.059620055s Dec 29 06:36:34.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.059455669s Dec 29 06:36:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.059519309s Dec 29 06:36:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.059760648s Dec 29 06:36:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.059800881s Dec 29 06:36:42.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.060586261s Dec 29 06:36:44.367: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.061701929s Dec 29 06:36:46.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.060203618s Dec 29 06:36:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.060285499s Dec 29 06:36:50.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.060088901s Dec 29 06:36:52.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.060341708s Dec 29 06:36:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.060154558s Dec 29 06:36:56.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.059262857s Dec 29 06:36:58.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.059513913s Dec 29 06:37:00.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.059550604s Dec 29 06:37:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.059947868s Dec 29 06:37:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.059615012s Dec 29 06:37:06.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.059315931s Dec 29 06:37:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.059820923s Dec 29 06:37:10.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.059057679s Dec 29 06:37:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.059568549s Dec 29 06:37:14.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.059699181s Dec 29 06:37:16.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.060824408s Dec 29 06:37:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.060256881s Dec 29 06:37:20.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.059355683s Dec 29 06:37:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.059892354s Dec 29 06:37:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.059887002s Dec 29 06:37:26.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.059145949s Dec 29 06:37:28.367: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.061476166s Dec 29 06:37:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.059912304s Dec 29 06:37:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.059924734s Dec 29 06:37:34.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.059288212s Dec 29 06:37:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.060130879s Dec 29 06:37:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.059731858s Dec 29 06:37:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.059858991s Dec 29 06:37:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.059957172s Dec 29 06:37:44.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.059811136s Dec 29 06:37:46.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.060312614s Dec 29 06:37:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.059973056s Dec 29 06:37:50.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.059309596s Dec 29 06:37:52.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.06014985s Dec 29 06:37:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.059422242s Dec 29 06:37:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.059514387s Dec 29 06:37:58.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.059325864s Dec 29 06:38:00.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.059709563s Dec 29 06:38:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.059642508s Dec 29 06:38:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.059596467s Dec 29 06:38:06.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.060775928s Dec 29 06:38:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.059413105s Dec 29 06:38:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.059868372s Dec 29 06:38:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.060181589s Dec 29 06:38:14.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.05973431s Dec 29 06:38:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.060267398s Dec 29 06:38:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.060185898s Dec 29 06:38:20.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.060133138s Dec 29 06:38:22.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.059066366s Dec 29 06:38:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.059924635s Dec 29 06:38:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.059362071s Dec 29 06:38:28.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.060168175s Dec 29 06:38:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.060110971s Dec 29 06:38:32.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.0591791s Dec 29 06:38:34.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.059623103s Dec 29 06:38:36.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.059111954s Dec 29 06:38:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.059975701s Dec 29 06:38:40.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.058488201s Dec 29 06:38:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.059553324s Dec 29 06:38:44.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.059070476s Dec 29 06:38:46.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.059270692s Dec 29 06:38:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.059720596s Dec 29 06:38:50.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.05910659s Dec 29 06:38:52.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.059831638s Dec 29 06:38:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.060040239s Dec 29 06:38:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.059855551s Dec 29 06:38:58.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m2.059656415s Dec 29 06:39:00.370: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m4.06473609s Dec 29 06:39:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m6.059528323s Dec 29 06:39:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m8.059437323s Dec 29 06:39:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m10.060219248s Dec 29 06:39:08.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m12.059094714s Dec 29 06:39:10.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m14.058973161s Dec 29 06:39:12.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m16.058550674s Dec 29 06:39:14.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m18.059709761s Dec 29 06:39:16.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m20.058477449s Dec 29 06:39:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m22.059790317s Dec 29 06:39:20.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m24.059073891s Dec 29 06:39:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m26.059475469s Dec 29 06:39:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m28.059665975s Dec 29 06:39:26.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m30.058670853s Dec 29 06:39:28.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m32.059909573s Dec 29 06:39:30.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m34.060686656s Dec 29 06:39:32.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m36.060524526s Dec 29 06:39:34.381: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m38.075603625s Dec 29 06:39:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m40.059736686s Dec 29 06:39:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m42.060097346s Dec 29 06:39:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m44.059722103s Dec 29 06:39:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m46.059893243s Dec 29 06:39:44.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m48.059883133s Dec 29 06:39:46.369: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m50.06345149s Dec 29 06:39:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m52.05959692s Dec 29 06:39:50.367: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m54.061443236s Dec 29 06:39:52.367: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m56.061835255s Dec 29 06:39:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 5m58.060007906s Dec 29 06:39:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m0.059848382s Dec 29 06:39:58.367: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m2.061590316s Dec 29 06:40:00.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m4.05951873s Dec 29 06:40:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m6.060240409s Dec 29 06:40:04.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m8.059205673s Dec 29 06:40:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m10.059754549s Dec 29 06:40:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m12.060234235s Dec 29 06:40:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m14.059676547s Dec 29 06:40:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m16.059560667s Dec 29 06:40:14.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m18.060249897s Dec 29 06:40:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m20.059745888s Dec 29 06:40:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m22.059838873s Dec 29 06:40:20.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m24.059729045s Dec 29 06:40:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m26.059877125s Dec 29 06:40:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m28.059979147s Dec 29 06:40:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m30.059824358s Dec 29 06:40:28.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m32.059659677s Dec 29 06:40:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m34.059528639s Dec 29 06:40:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m36.059706365s Dec 29 06:40:34.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m38.059654675s Dec 29 06:40:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m40.059672169s Dec 29 06:40:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m42.060043121s Dec 29 06:40:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m44.059904792s Dec 29 06:40:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m46.060014707s Dec 29 06:40:44.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m48.059680132s Dec 29 06:40:46.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m50.059682176s Dec 29 06:40:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m52.05990402s Dec 29 06:40:50.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m54.059956363s Dec 29 06:40:52.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m56.059236207s Dec 29 06:40:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 6m58.059487527s Dec 29 06:40:56.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m0.058937631s Dec 29 06:40:58.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m2.060103491s Dec 29 06:41:00.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m4.059480686s Dec 29 06:41:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m6.059392048s Dec 29 06:41:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m8.059840262s Dec 29 06:41:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m10.060039569s Dec 29 06:41:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m12.059767902s Dec 29 06:41:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m14.059915838s Dec 29 06:41:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m16.059674333s Dec 29 06:41:14.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m18.059251966s Dec 29 06:41:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m20.060315705s Dec 29 06:41:18.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m22.059220068s Dec 29 06:41:20.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m24.059264053s Dec 29 06:41:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m26.05935567s Dec 29 06:41:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m28.060083825s Dec 29 06:41:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m30.060245229s Dec 29 06:41:28.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m32.060333897s Dec 29 06:41:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m34.059935176s Dec 29 06:41:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m36.060153159s Dec 29 06:41:34.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m38.059198291s Dec 29 06:41:36.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m40.060575546s Dec 29 06:41:38.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m42.060504772s Dec 29 06:41:40.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m44.059258391s Dec 29 06:41:42.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m46.060767918s Dec 29 06:41:44.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m48.059819511s Dec 29 06:41:46.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m50.060048118s Dec 29 06:41:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m52.059597685s Dec 29 06:41:50.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m54.058870226s Dec 29 06:41:52.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m56.059692747s Dec 29 06:41:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 7m58.060198164s Dec 29 06:41:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m0.05960544s Dec 29 06:41:58.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m2.061341976s Dec 29 06:42:00.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m4.059311382s Dec 29 06:42:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m6.059797999s Dec 29 06:42:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m8.059695151s Dec 29 06:42:06.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m10.059312482s Dec 29 06:42:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m12.059886443s Dec 29 06:42:10.370: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m14.064843407s Dec 29 06:42:12.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m16.059074735s Dec 29 06:42:14.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m18.059586979s Dec 29 06:42:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m20.059513248s Dec 29 06:42:18.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m22.060234377s Dec 29 06:42:20.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m24.060290738s Dec 29 06:42:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m26.059855172s Dec 29 06:42:24.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m28.059875033s Dec 29 06:42:26.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m30.059058341s Dec 29 06:42:28.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m32.060287777s Dec 29 06:42:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m34.059457629s Dec 29 06:42:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m36.05981382s Dec 29 06:42:34.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m38.059223139s Dec 29 06:42:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m40.060126228s Dec 29 06:42:38.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m42.060400617s Dec 29 06:42:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m44.060062406s Dec 29 06:42:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m46.059611874s Dec 29 06:42:44.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m48.060756241s Dec 29 06:42:46.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m50.058953719s Dec 29 06:42:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m52.059595997s Dec 29 06:42:50.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m54.061272283s Dec 29 06:42:52.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m56.059525341s Dec 29 06:42:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 8m58.059417658s Dec 29 06:42:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m0.059594985s Dec 29 06:42:58.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m2.060325218s Dec 29 06:43:00.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m4.059948319s Dec 29 06:43:02.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m6.059738103s Dec 29 06:43:04.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m8.05974789s Dec 29 06:43:06.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m10.059400578s Dec 29 06:43:08.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m12.059509915s Dec 29 06:43:10.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m14.059576066s Dec 29 06:43:12.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m16.059784919s Dec 29 06:43:14.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m18.059399861s Dec 29 06:43:16.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m20.059896702s Dec 29 06:43:18.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m22.060349682s Dec 29 06:43:20.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m24.060209471s Dec 29 06:43:22.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m26.060073788s Dec 29 06:43:24.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m28.059170279s Dec 29 06:43:26.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m30.059902392s Dec 29 06:43:28.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m32.059132632s Dec 29 06:43:30.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m34.059472257s Dec 29 06:43:32.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m36.059570042s Dec 29 06:43:34.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m38.059350239s Dec 29 06:43:36.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m40.059790516s Dec 29 06:43:38.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m42.060034397s Dec 29 06:43:40.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m44.059634763s Dec 29 06:43:42.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m46.059764179s Dec 29 06:43:44.366: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m48.06056079s Dec 29 06:43:46.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m50.060302085s Dec 29 06:43:48.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m52.059478044s Dec 29 06:43:50.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m54.060107263s Dec 29 06:43:52.364: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m56.059228772s Dec 29 06:43:54.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 9m58.060137662s Dec 29 06:43:56.365: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 10m0.059991937s Dec 29 06:43:56.395: INFO: Pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351": Phase="Pending", Reason="", readiness=false. Elapsed: 10m0.089445895s Dec 29 06:43:56.395: INFO: Unexpected error: while recreating pod for resizing: <*errors.errorString | 0xc0016070d0>: { s: "pod \"pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351\" is not Running: timed out while waiting for pod volume-expand-5953/pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 to be running", } Dec 29 06:43:56.395: FAIL: while recreating pod for resizing: pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351" is not Running: timed out while waiting for pod volume-expand-5953/pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeExpandTestSuite).DefineTests.func4() test/e2e/storage/testsuites/volume_expand.go:238 +0xe5a Dec 29 06:43:56.395: INFO: Deleting pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351" in namespace "volume-expand-5953" Dec 29 06:43:56.429: INFO: Wait up to 5m0s for pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351" to be fully deleted Dec 29 06:43:58.489: INFO: Deleting pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe" in namespace "volume-expand-5953" �[1mSTEP:�[0m Deleting pod �[38;5;243m12/29/22 06:43:58.518�[0m Dec 29 06:43:58.518: INFO: Deleting pod "pod-604e9e03-46a2-4b9f-8312-7c14684f49fe" in namespace "volume-expand-5953" �[1mSTEP:�[0m Deleting pod2 �[38;5;243m12/29/22 06:43:58.548�[0m Dec 29 06:43:58.548: INFO: Deleting pod "pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351" in namespace "volume-expand-5953" �[1mSTEP:�[0m Deleting pvc �[38;5;243m12/29/22 06:43:58.577�[0m Dec 29 06:43:58.635: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.compktvk" Dec 29 06:43:58.667: INFO: Waiting up to 5m0s for PersistentVolume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 to get deleted Dec 29 06:43:58.696: INFO: PersistentVolume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 found and phase=Released (29.210525ms) Dec 29 06:44:03.727: INFO: PersistentVolume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 found and phase=Released (5.060121937s) Dec 29 06:44:08.757: INFO: PersistentVolume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 found and phase=Released (10.090563843s) Dec 29 06:44:13.788: INFO: PersistentVolume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 found and phase=Released (15.121207659s) Dec 29 06:44:18.818: INFO: PersistentVolume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 was removed �[1mSTEP:�[0m Deleting sc �[38;5;243m12/29/22 06:44:18.818�[0m [AfterEach] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand test/e2e/framework/node/init/init.go:32 Dec 29 06:44:18.850: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:44:18.881�[0m �[1mSTEP:�[0m Collecting events from namespace "volume-expand-5953". �[38;5;243m12/29/22 06:44:18.881�[0m �[1mSTEP:�[0m Found 26 events. �[38;5;243m12/29/22 06:44:18.913�[0m Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:25 +0000 UTC - event for ebs.csi.aws.compktvk: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:25 +0000 UTC - event for ebs.csi.aws.compktvk: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-j8vxm_47cfaf09-8c13-4f52-9a11-ea60ac01cfa3 } Provisioning: External provisioner is provisioning volume for claim "volume-expand-5953/ebs.csi.aws.compktvk" Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:25 +0000 UTC - event for ebs.csi.aws.compktvk: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:29 +0000 UTC - event for ebs.csi.aws.compktvk: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-j8vxm_47cfaf09-8c13-4f52-9a11-ea60ac01cfa3 } ProvisioningSucceeded: Successfully provisioned volume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:29 +0000 UTC - event for pod-604e9e03-46a2-4b9f-8312-7c14684f49fe: {default-scheduler } Scheduled: Successfully assigned volume-expand-5953/pod-604e9e03-46a2-4b9f-8312-7c14684f49fe to i-0b6ab0d2cddb81acd Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:32 +0000 UTC - event for pod-604e9e03-46a2-4b9f-8312-7c14684f49fe: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14" Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:40 +0000 UTC - event for pod-604e9e03-46a2-4b9f-8312-7c14684f49fe: {kubelet i-0b6ab0d2cddb81acd} Started: Started container write-pod Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:40 +0000 UTC - event for pod-604e9e03-46a2-4b9f-8312-7c14684f49fe: {kubelet i-0b6ab0d2cddb81acd} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:40 +0000 UTC - event for pod-604e9e03-46a2-4b9f-8312-7c14684f49fe: {kubelet i-0b6ab0d2cddb81acd} Created: Created container write-pod Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:48 +0000 UTC - event for pod-604e9e03-46a2-4b9f-8312-7c14684f49fe: {kubelet i-0b6ab0d2cddb81acd} Killing: Stopping container write-pod Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:52 +0000 UTC - event for ebs.csi.aws.compktvk: {volume_expand } ExternalExpanding: Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC. Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:52 +0000 UTC - event for ebs.csi.aws.compktvk: {external-resizer ebs.csi.aws.com } Resizing: External resizer is resizing volume pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14 Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:55 +0000 UTC - event for ebs.csi.aws.compktvk: {external-resizer ebs.csi.aws.com } FileSystemResizeRequired: Require file system resize of volume on node Dec 29 06:44:18.913: INFO: At 2022-12-29 06:33:56 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {default-scheduler } Scheduled: Successfully assigned volume-expand-5953/pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 to i-0b6ab0d2cddb81acd Dec 29 06:44:18.913: INFO: At 2022-12-29 06:34:12 +0000 UTC - event for ebs.csi.aws.compktvk: {kubelet i-0b6ab0d2cddb81acd} FileSystemResizeSuccessful: MountVolume.NodeExpandVolume succeeded for volume "pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14" i-0b6ab0d2cddb81acd Dec 29 06:44:18.913: INFO: At 2022-12-29 06:34:12 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FileSystemResizeSuccessful: MountVolume.NodeExpandVolume succeeded for volume "pvc-6c997b6f-b9fa-4de2-8ee8-e86f3be4ae14" i-0b6ab0d2cddb81acd Dec 29 06:44:18.913: INFO: At 2022-12-29 06:34:12 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "66c0912510d111bdcea40cc4f5a6be793a480bfc54352d22900c1ea1db7a879b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:34:24 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "32150b3e4633240fd7fdb42c242a8f5d9bb3f0c663c7f9509c06ea083ce8c39b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:34:37 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "359070724b0d07073705b7a4cf7c453ba2f4118a7f3c1b4e459e122ff40d449f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:34:51 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "17cc112c7345009684d1cbe8e370bf49c81d9c1789994535736b654d9c994014": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:35:08 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b41565a470e3be384a80bcd77c621a9eff674157218c0f246955dbd2693928bc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:35:18 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e5df6b5a6df88b78662c1591ac517d0f9eea1039148a96331e01d4ef1ad73fae": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:35:33 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "98c68f86799a9f6fb1adfe2a3da3531fe08918d8789611ff424e4c7e9f2be446": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:35:45 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "09ebac89f47d1a0bc1e583201c625df70ad0a0da174cba70873f1fe00b927b09": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:35:56 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0179712ec1498726a41f97c8c612891aac25704812db3ae106d55557d727250f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.913: INFO: At 2022-12-29 06:36:07 +0000 UTC - event for pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7901363c99e8ef736becc7f37172335ce7f5ba6b358ed91be26b3477841c7a2e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:18.943: INFO: POD NODE PHASE GRACE CONDITIONS Dec 29 06:44:18.943: INFO: Dec 29 06:44:18.973: INFO: Logging node info for node i-0345e6864820ace32 Dec 29 06:44:19.003: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 21806 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799,DevicePath:,},},Config:nil,},} Dec 29 06:44:19.003: INFO: Logging kubelet events for node i-0345e6864820ace32 Dec 29 06:44:19.035: INFO: Logging pods the kubelet thinks is on node i-0345e6864820ace32 Dec 29 06:44:19.100: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:19.100: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:19.100: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container webserver ready: true, restart count 0 Dec 29 06:44:19.100: INFO: hostexec-i-0345e6864820ace32-29fpj started at 2022-12-29 06:42:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:44:19.100: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.100: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:19.100: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:19.100: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dwftm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvtcq started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container coredns ready: true, restart count 0 Dec 29 06:44:19.100: INFO: pod-subpath-test-preprovisionedpv-5b4m started at 2022-12-29 06:42:39 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Init container init-volume-preprovisionedpv-5b4m ready: false, restart count 0 Dec 29 06:44:19.100: INFO: Container test-container-subpath-preprovisionedpv-5b4m ready: false, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: netserver-0 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gtk84 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: inline-volume-tester2-lkk7j started at 2022-12-29 06:41:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container csi-volume-tester ready: false, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-ck56n started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.100: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-mcnp2 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.100: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.281: INFO: Latency metrics for node i-0345e6864820ace32 Dec 29 06:44:19.281: INFO: Logging node info for node i-06be1dd8c97acf782 Dec 29 06:44:19.311: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 21540 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:41:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:44:19.312: INFO: Logging kubelet events for node i-06be1dd8c97acf782 Dec 29 06:44:19.344: INFO: Logging pods the kubelet thinks is on node i-06be1dd8c97acf782 Dec 29 06:44:19.380: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.380: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:19.380: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container kops-controller ready: true, restart count 0 Dec 29 06:44:19.380: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded) Dec 29 06:44:19.380: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:19.380: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:44:19.380: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded) Dec 29 06:44:19.380: INFO: Container healthcheck ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container kube-apiserver ready: true, restart count 1 Dec 29 06:44:19.380: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container kube-controller-manager ready: true, restart count 2 Dec 29 06:44:19.380: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:19.380: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:19.380: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Dec 29 06:44:19.380: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:44:19.380: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container kube-scheduler ready: true, restart count 0 Dec 29 06:44:19.380: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container dns-controller ready: true, restart count 0 Dec 29 06:44:19.380: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.380: INFO: Container cilium-operator ready: true, restart count 0 Dec 29 06:44:19.536: INFO: Latency metrics for node i-06be1dd8c97acf782 Dec 29 06:44:19.536: INFO: Logging node info for node i-07228a57ea47d842b Dec 29 06:44:19.567: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 20587 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9651^9428e159-8743-11ed-b070-a6882e2b2850],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:44:19.567: INFO: Logging kubelet events for node i-07228a57ea47d842b Dec 29 06:44:19.600: INFO: Logging pods the kubelet thinks is on node i-07228a57ea47d842b Dec 29 06:44:19.640: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b9h6m started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4xmqc started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container volume-tester ready: false, restart count 0 Dec 29 06:44:19.640: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.640: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:19.640: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:19.640: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:19.640: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.640: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:44:19.640: INFO: Container driver-registrar ready: true, restart count 0 Dec 29 06:44:19.640: INFO: Container mock ready: true, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4l6sw started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container autoscaler ready: true, restart count 0 Dec 29 06:44:19.640: INFO: hostexec-i-07228a57ea47d842b-9fc7b started at 2022-12-29 06:39:25 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-57b2m started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:44:19.640: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b6wc8 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: pod-592477cc-870d-47db-b10f-254f00e2207c started at 2022-12-29 06:39:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:44:19.640: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:19.640: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:19.640: INFO: pod-2526fd38-277f-499e-b2c5-b55842c84c2d started at 2022-12-29 06:39:55 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:44:19.640: INFO: netserver-1 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:19.640: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.640: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.640: INFO: Container coredns ready: true, restart count 0 Dec 29 06:44:19.825: INFO: Latency metrics for node i-07228a57ea47d842b Dec 29 06:44:19.825: INFO: Logging node info for node i-07d5c844ec1aba791 Dec 29 06:44:19.855: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 21778 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6856":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:41:58 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:41:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf,DevicePath:,},},Config:nil,},} Dec 29 06:44:19.855: INFO: Logging kubelet events for node i-07d5c844ec1aba791 Dec 29 06:44:19.887: INFO: Logging pods the kubelet thinks is on node i-07d5c844ec1aba791 Dec 29 06:44:19.927: INFO: pod-projected-configmaps-03cdd563-f67f-480c-8bad-23ae8f2d256a started at 2022-12-29 06:42:28 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.927: INFO: Container createcm-volume-test ready: false, restart count 0 Dec 29 06:44:19.927: INFO: Container delcm-volume-test ready: false, restart count 0 Dec 29 06:44:19.927: INFO: Container updcm-volume-test ready: false, restart count 0 Dec 29 06:44:19.927: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container webserver ready: true, restart count 0 Dec 29 06:44:19.927: INFO: pod-configmaps-191e839a-8743-4828-8230-52b8b3ce9a18 started at 2022-12-29 06:39:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container agnhost-container ready: false, restart count 0 Dec 29 06:44:19.927: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: busybox-3b39c30d-31e2-49ad-81c2-e1e37a94c517 started at 2022-12-29 06:39:50 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container busybox ready: false, restart count 0 Dec 29 06:44:19.927: INFO: pod-73af6e22-e126-427f-bec8-33101a9f6c91 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:33:49 +0000 UTC (0+7 container statuses recorded) Dec 29 06:44:19.927: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container csi-snapshotter ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container hostpath ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:19.927: INFO: failed-jobs-history-limit-27871602-87bxk started at 2022-12-29 06:42:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: failed-jobs-history-limit-27871603-sr5mg started at 2022-12-29 06:43:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: hostpath-symlink-prep-provisioning-3263 started at 2022-12-29 06:39:49 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container init-volume-provisioning-3263 ready: false, restart count 0 Dec 29 06:44:19.927: INFO: failed-jobs-history-limit-27871601-lg7kl started at 2022-12-29 06:41:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: hostpath-injector started at 2022-12-29 06:40:02 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container hostpath-injector ready: false, restart count 0 Dec 29 06:44:19.927: INFO: failed-jobs-history-limit-27871600-p8jnr started at 2022-12-29 06:40:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-j7svz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: failed-jobs-history-limit-27871604-jqgj9 started at 2022-12-29 06:44:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: pod-0 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container donothing ready: true, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zlzqg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:44:19.927: INFO: netserver-2 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:19.927: INFO: sample-webhook-deployment-865554f4d9-n7km6 started at 2022-12-29 06:41:01 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container sample-webhook ready: false, restart count 0 Dec 29 06:44:19.927: INFO: pod-1 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container donothing ready: false, restart count 0 Dec 29 06:44:19.927: INFO: httpd started at 2022-12-29 06:42:01 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container httpd ready: false, restart count 0 Dec 29 06:44:19.927: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.927: INFO: Container jessie-querier ready: false, restart count 0 Dec 29 06:44:19.927: INFO: Container querier ready: false, restart count 0 Dec 29 06:44:19.927: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:19.927: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container csi-volume-tester ready: true, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9v6vz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:19.927: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:19.927: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:19.927: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:19.927: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:20.145: INFO: Latency metrics for node i-07d5c844ec1aba791 Dec 29 06:44:20.145: INFO: Logging node info for node i-0b6ab0d2cddb81acd Dec 29 06:44:20.175: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 22329 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:44:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2022-12-29 06:44:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:00 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:00 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:00 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:00 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420,DevicePath:,},},Config:nil,},} Dec 29 06:44:20.175: INFO: Logging kubelet events for node i-0b6ab0d2cddb81acd Dec 29 06:44:20.207: INFO: Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6h6vh started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: hostexec-i-0b6ab0d2cddb81acd-qppr8 started at 2022-12-29 06:34:30 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container donothing ready: true, restart count 0 Dec 29 06:44:20.245: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:20.245: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:44:20.245: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:44:20.245: INFO: Container mock ready: false, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: pod-subpath-test-dynamicpv-t84v started at 2022-12-29 06:39:29 +0000 UTC (0+2 container statuses recorded) Dec 29 06:44:20.245: INFO: Container test-container-subpath-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:44:20.245: INFO: Container test-container-volume-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:44:20.245: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:44:20.245: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:20.245: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:20.245: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:20.245: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:20.245: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:20.245: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:20.245: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d8lgx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-8qfzg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:20.245: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:20.245: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:44:20.245: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:44:20.245: INFO: Container mock ready: false, restart count 0 Dec 29 06:44:20.245: INFO: local-client started at 2022-12-29 06:39:39 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container local-client ready: false, restart count 0 Dec 29 06:44:20.245: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:20.245: INFO: netserver-3 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:20.245: INFO: downwardapi-volume-cc6aea7e-266b-4611-a490-6cb3559eaf08 started at 2022-12-29 06:39:43 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:20.245: INFO: Container client-container ready: false, restart count 0 Dec 29 06:44:20.474: INFO: Latency metrics for node i-0b6ab0d2cddb81acd [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "volume-expand-5953" for this suite. �[38;5;243m12/29/22 06:44:20.474�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\sbe\sable\sto\sunmount\safter\sthe\ssubpath\sdirectory\sis\sdeleted\s\[LinuxOnly\]$'
test/e2e/storage/testsuites/subpath.go:465 k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func20() test/e2e/storage/testsuites/subpath.go:465 +0x405from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:39:25.365�[0m Dec 29 06:39:25.365: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename provisioning �[38;5;243m12/29/22 06:39:25.367�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:39:25.457�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:39:25.515�[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/metrics/init/init.go:31 [It] should be able to unmount after the subpath directory is deleted [LinuxOnly] test/e2e/storage/testsuites/subpath.go:445 Dec 29 06:39:25.575: INFO: Creating resource for dynamic PV Dec 29 06:39:25.575: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP:�[0m creating a StorageClass provisioning-7696-e2e-sclj942 �[38;5;243m12/29/22 06:39:25.576�[0m �[1mSTEP:�[0m creating a claim �[38;5;243m12/29/22 06:39:25.637�[0m Dec 29 06:39:25.637: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP:�[0m Creating pod pod-subpath-test-dynamicpv-t84v �[38;5;243m12/29/22 06:39:25.716�[0m Dec 29 06:39:25.752: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-t84v" in namespace "provisioning-7696" to be "running" Dec 29 06:39:25.789: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 37.02606ms Dec 29 06:39:27.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2.067923557s Dec 29 06:39:29.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4.067685692s Dec 29 06:39:31.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 6.068231369s Dec 29 06:39:33.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 8.066861038s Dec 29 06:39:35.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 10.06782491s Dec 29 06:39:37.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 12.066983005s Dec 29 06:39:39.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 14.067655495s Dec 29 06:39:41.822: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 16.07001593s Dec 29 06:39:43.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 18.068024803s Dec 29 06:39:45.821: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 20.069050076s Dec 29 06:39:47.818: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 22.06663177s Dec 29 06:39:49.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 24.067120815s Dec 29 06:39:51.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 26.066988283s Dec 29 06:39:53.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 28.067120065s Dec 29 06:39:55.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 30.06678387s Dec 29 06:39:57.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 32.067266744s Dec 29 06:39:59.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 34.067038464s Dec 29 06:40:01.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 36.066886247s Dec 29 06:40:03.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 38.067892728s Dec 29 06:40:05.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 40.067451829s Dec 29 06:40:07.821: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 42.069041147s Dec 29 06:40:09.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 44.068046051s Dec 29 06:40:11.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 46.067700873s Dec 29 06:40:13.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 48.067700778s Dec 29 06:40:15.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 50.067529433s Dec 29 06:40:17.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 52.066957856s Dec 29 06:40:19.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 54.067861923s Dec 29 06:40:21.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 56.067321192s Dec 29 06:40:23.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 58.06746541s Dec 29 06:40:25.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.066975168s Dec 29 06:40:27.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.067351828s Dec 29 06:40:29.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.06794567s Dec 29 06:40:31.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.067409569s Dec 29 06:40:33.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.06772189s Dec 29 06:40:35.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.067489415s Dec 29 06:40:37.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.067146389s Dec 29 06:40:39.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.068177202s Dec 29 06:40:41.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.067465081s Dec 29 06:40:43.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.068003115s Dec 29 06:40:45.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.067206987s Dec 29 06:40:47.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.067658322s Dec 29 06:40:49.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.067831189s Dec 29 06:40:51.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.067963919s Dec 29 06:40:53.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.067367801s Dec 29 06:40:55.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.067287536s Dec 29 06:40:57.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.067942779s Dec 29 06:40:59.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.067435181s Dec 29 06:41:01.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.06801064s Dec 29 06:41:03.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.068229432s Dec 29 06:41:05.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.067000281s Dec 29 06:41:07.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.067908922s Dec 29 06:41:09.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.06756569s Dec 29 06:41:11.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.067629401s Dec 29 06:41:13.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.067907057s Dec 29 06:41:15.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.067414668s Dec 29 06:41:17.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.067942583s Dec 29 06:41:19.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.067745868s Dec 29 06:41:21.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.067971935s Dec 29 06:41:23.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.067791346s Dec 29 06:41:25.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.067116433s Dec 29 06:41:27.818: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.066725344s Dec 29 06:41:29.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.068531232s Dec 29 06:41:31.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.067106446s Dec 29 06:41:33.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.067823135s Dec 29 06:41:35.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.067524242s Dec 29 06:41:37.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.068113056s Dec 29 06:41:39.821: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.069729759s Dec 29 06:41:41.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.067384508s Dec 29 06:41:43.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.067366407s Dec 29 06:41:45.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.068299164s Dec 29 06:41:47.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.067067287s Dec 29 06:41:49.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.067394742s Dec 29 06:41:51.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.068351741s Dec 29 06:41:53.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.067633155s Dec 29 06:41:55.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.067772376s Dec 29 06:41:57.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.067668448s Dec 29 06:41:59.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.068178514s Dec 29 06:42:01.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.067862387s Dec 29 06:42:03.818: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.066739154s Dec 29 06:42:05.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.067632752s Dec 29 06:42:07.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.067115484s Dec 29 06:42:09.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.067718304s Dec 29 06:42:11.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.067312596s Dec 29 06:42:13.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.068186178s Dec 29 06:42:15.822: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.070686649s Dec 29 06:42:17.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.068229912s Dec 29 06:42:19.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.067279148s Dec 29 06:42:21.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.066885856s Dec 29 06:42:23.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.067573067s Dec 29 06:42:25.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.067865008s Dec 29 06:42:27.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.067942436s Dec 29 06:42:29.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.067337473s Dec 29 06:42:31.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.06795637s Dec 29 06:42:33.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.068273696s Dec 29 06:42:35.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.067471611s Dec 29 06:42:37.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.06787601s Dec 29 06:42:39.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.067889347s Dec 29 06:42:41.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.067483638s Dec 29 06:42:43.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.067806278s Dec 29 06:42:45.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.067412996s Dec 29 06:42:47.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.06730682s Dec 29 06:42:49.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.068412829s Dec 29 06:42:51.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.066964686s Dec 29 06:42:53.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.067270549s Dec 29 06:42:55.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.067020654s Dec 29 06:42:57.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.067846268s Dec 29 06:42:59.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.066947215s Dec 29 06:43:01.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.068221251s Dec 29 06:43:03.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.06842938s Dec 29 06:43:05.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.067654306s Dec 29 06:43:07.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.067694817s Dec 29 06:43:09.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.06786949s Dec 29 06:43:11.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.067803877s Dec 29 06:43:13.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.067627371s Dec 29 06:43:15.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.067331318s Dec 29 06:43:17.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.068369219s Dec 29 06:43:19.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.067999237s Dec 29 06:43:21.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.067722002s Dec 29 06:43:23.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.067278063s Dec 29 06:43:25.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.067373158s Dec 29 06:43:27.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.067057995s Dec 29 06:43:29.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.06715656s Dec 29 06:43:31.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.068118273s Dec 29 06:43:33.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.067656654s Dec 29 06:43:35.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.067438139s Dec 29 06:43:37.821: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.069251589s Dec 29 06:43:39.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.067921752s Dec 29 06:43:41.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.067884008s Dec 29 06:43:43.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.068136357s Dec 29 06:43:45.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.067118308s Dec 29 06:43:47.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.067846295s Dec 29 06:43:49.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.067919908s Dec 29 06:43:51.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.067580316s Dec 29 06:43:53.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.067357908s Dec 29 06:43:55.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.068468979s Dec 29 06:43:57.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.067267574s Dec 29 06:43:59.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.067119778s Dec 29 06:44:01.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.068241873s Dec 29 06:44:03.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.067622695s Dec 29 06:44:05.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.067252333s Dec 29 06:44:07.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.067042237s Dec 29 06:44:09.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.067706032s Dec 29 06:44:11.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.067055681s Dec 29 06:44:13.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.068140215s Dec 29 06:44:15.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.067321173s Dec 29 06:44:17.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.068579148s Dec 29 06:44:19.820: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.067857111s Dec 29 06:44:21.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.067169844s Dec 29 06:44:23.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.066948891s Dec 29 06:44:25.819: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.06701768s Dec 29 06:44:25.851: INFO: Pod "pod-subpath-test-dynamicpv-t84v": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.098757145s Dec 29 06:44:25.852: INFO: Unexpected error: while waiting for pod to be running: <*pod.timeoutError | 0xc0070ab830>: { msg: "timed out while waiting for pod provisioning-7696/pod-subpath-test-dynamicpv-t84v to be running", observedObjects: [ <*v1.Pod | 0xc002c0b200>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-subpath-test-dynamicpv-t84v", GenerateName: "", Namespace: "provisioning-7696", SelfLink: "", UID: "156272be-5cda-4f7f-9b5e-b0430f395ed7", ResourceVersion: "19785", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63807892765, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63807892765, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"test-container-subpath-dynamicpv-t84v\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/probe-volume\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}},\"k:{\\\"mountPath\\\":\\\"/test-volume\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{},\"f:subPath\":{}}}},\"k:{\\\"name\\\":\\\"test-container-volume-dynamicpv-t84v\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/probe-volume\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}},\"k:{\\\"mountPath\\\":\\\"/test-volume\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:seLinuxOptions\":{\".\":{},\"f:level\":{}}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"liveness-probe-volume\\\"}\":{\".\":{},\"f:emptyDir\":{},\"f:name\":{}},\"k:{\\\"name\\\":\\\"test-volume\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63807892769, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "test-volume", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "ebs.csi.aws.comkdlnr", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 29 06:44:25.852: FAIL: while waiting for pod to be running: timed out while waiting for pod provisioning-7696/pod-subpath-test-dynamicpv-t84v to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func20() test/e2e/storage/testsuites/subpath.go:465 +0x405 �[1mSTEP:�[0m Deleting pod pod-subpath-test-dynamicpv-t84v �[38;5;243m12/29/22 06:44:25.852�[0m Dec 29 06:44:25.852: INFO: Deleting pod "pod-subpath-test-dynamicpv-t84v" in namespace "provisioning-7696" Dec 29 06:44:25.887: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-t84v" to be fully deleted �[1mSTEP:�[0m Deleting pod �[38;5;243m12/29/22 06:44:27.947�[0m Dec 29 06:44:27.947: INFO: Deleting pod "pod-subpath-test-dynamicpv-t84v" in namespace "provisioning-7696" �[1mSTEP:�[0m Deleting pvc �[38;5;243m12/29/22 06:44:27.977�[0m Dec 29 06:44:28.036: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comkdlnr" Dec 29 06:44:28.069: INFO: Waiting up to 5m0s for PersistentVolume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 to get deleted Dec 29 06:44:28.098: INFO: PersistentVolume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 found and phase=Released (29.483451ms) Dec 29 06:44:33.136: INFO: PersistentVolume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 found and phase=Released (5.067338595s) Dec 29 06:44:38.166: INFO: PersistentVolume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 found and phase=Released (10.097548178s) Dec 29 06:44:43.198: INFO: PersistentVolume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 found and phase=Released (15.129308907s) Dec 29 06:44:48.229: INFO: PersistentVolume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 was removed �[1mSTEP:�[0m Deleting sc �[38;5;243m12/29/22 06:44:48.229�[0m [AfterEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/node/init/init.go:32 Dec 29 06:44:48.261: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:44:48.292�[0m �[1mSTEP:�[0m Collecting events from namespace "provisioning-7696". �[38;5;243m12/29/22 06:44:48.292�[0m �[1mSTEP:�[0m Found 16 events. �[38;5;243m12/29/22 06:44:48.322�[0m Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:25 +0000 UTC - event for ebs.csi.aws.comkdlnr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:25 +0000 UTC - event for ebs.csi.aws.comkdlnr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:25 +0000 UTC - event for ebs.csi.aws.comkdlnr: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-j8vxm_47cfaf09-8c13-4f52-9a11-ea60ac01cfa3 } Provisioning: External provisioner is provisioning volume for claim "provisioning-7696/ebs.csi.aws.comkdlnr" Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:29 +0000 UTC - event for ebs.csi.aws.comkdlnr: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-j8vxm_47cfaf09-8c13-4f52-9a11-ea60ac01cfa3 } ProvisioningSucceeded: Successfully provisioned volume pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1 Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:29 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {default-scheduler } Scheduled: Successfully assigned provisioning-7696/pod-subpath-test-dynamicpv-t84v to i-0b6ab0d2cddb81acd Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:32 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-bdc598d3-a527-4fe3-bbd6-5e34a79900d1" Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:37 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "44c72ed6303eb0453be82647985160b2813c204c6c7b4b16a85097d478f6b4b6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:39:50 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3ecb43a349fd8e029255c30912331690a5fd99a05c1226bf984dcfe49a616ecb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:40:04 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0b4700f786c814e7b38be32428a32f1952726827f7869248163a22615d947cee": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:40:17 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "68ef3f9c4fe992cc4a91d2dd1e5a01422aa53ea0f1893ef97491f007baca5fd1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:40:29 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6d0354c6d791dceab5bce31fe7a1dd57f87c961a855ed291131919e0e7fe303a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:40:39 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2d6c463df46d84fee3ff9f9ac99ac1d3fb1e5c249ddb705e2bb77fbd7c2c95bd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:40:50 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "646ef3f370e469323ed14bcb92e77a2421e5bc10188d22a55bdf767327a94b74": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:41:03 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7137421632dd83825030a91ec96f86d5f363f0b385d33da525a9ab65b4676f1c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:41:15 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "99d7a0e058e530910f20595b1732b32014fb5a3ee5c5f4d3cee4045266402c53": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.323: INFO: At 2022-12-29 06:41:30 +0000 UTC - event for pod-subpath-test-dynamicpv-t84v: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "831a1e44d9bd3a88da042712aa35ae4fe5b9c300543c5bb0081cc5670d8b010d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:44:48.352: INFO: POD NODE PHASE GRACE CONDITIONS Dec 29 06:44:48.352: INFO: Dec 29 06:44:48.391: INFO: Logging node info for node i-0345e6864820ace32 Dec 29 06:44:48.427: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 22567 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799,DevicePath:,},},Config:nil,},} Dec 29 06:44:48.428: INFO: Logging kubelet events for node i-0345e6864820ace32 Dec 29 06:44:48.460: INFO: Logging pods the kubelet thinks is on node i-0345e6864820ace32 Dec 29 06:44:48.498: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container coredns ready: true, restart count 0 Dec 29 06:44:48.498: INFO: pod-subpath-test-preprovisionedpv-5b4m started at 2022-12-29 06:42:39 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Init container init-volume-preprovisionedpv-5b4m ready: false, restart count 0 Dec 29 06:44:48.498: INFO: Container test-container-subpath-preprovisionedpv-5b4m ready: false, restart count 0 Dec 29 06:44:48.498: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:48.498: INFO: netserver-0 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:48.498: INFO: netserver-0 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:48.498: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:48.498: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:48.498: INFO: inline-volume-tester2-lkk7j started at 2022-12-29 06:41:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container csi-volume-tester ready: false, restart count 0 Dec 29 06:44:48.498: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:48.498: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:48.498: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container webserver ready: true, restart count 0 Dec 29 06:44:48.498: INFO: hostexec-i-0345e6864820ace32-29fpj started at 2022-12-29 06:42:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:44:48.498: INFO: simpletest.rc-gdhwc started at 2022-12-29 06:44:20 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container nginx ready: false, restart count 0 Dec 29 06:44:48.498: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:48.498: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:48.498: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:48.498: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:48.498: INFO: netserver-0 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.498: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:48.793: INFO: Latency metrics for node i-0345e6864820ace32 Dec 29 06:44:48.793: INFO: Logging node info for node i-06be1dd8c97acf782 Dec 29 06:44:48.825: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 21540 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:41:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:44:48.825: INFO: Logging kubelet events for node i-06be1dd8c97acf782 Dec 29 06:44:48.860: INFO: Logging pods the kubelet thinks is on node i-06be1dd8c97acf782 Dec 29 06:44:48.907: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container kube-scheduler ready: true, restart count 0 Dec 29 06:44:48.907: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container dns-controller ready: true, restart count 0 Dec 29 06:44:48.907: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container cilium-operator ready: true, restart count 0 Dec 29 06:44:48.907: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Dec 29 06:44:48.907: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:44:48.907: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded) Dec 29 06:44:48.907: INFO: Container healthcheck ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container kube-apiserver ready: true, restart count 1 Dec 29 06:44:48.907: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container kube-controller-manager ready: true, restart count 2 Dec 29 06:44:48.907: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:48.907: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:48.907: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:48.907: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container kops-controller ready: true, restart count 0 Dec 29 06:44:48.907: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded) Dec 29 06:44:48.907: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:48.907: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:48.907: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:48.907: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:44:49.098: INFO: Latency metrics for node i-06be1dd8c97acf782 Dec 29 06:44:49.098: INFO: Logging node info for node i-07228a57ea47d842b Dec 29 06:44:49.128: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 22569 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2022-12-29 06:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9651^9428e159-8743-11ed-b070-a6882e2b2850],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:44:49.128: INFO: Logging kubelet events for node i-07228a57ea47d842b Dec 29 06:44:49.161: INFO: Logging pods the kubelet thinks is on node i-07228a57ea47d842b Dec 29 06:44:49.200: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.200: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:49.200: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:49.200: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:49.200: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container volume-tester ready: false, restart count 0 Dec 29 06:44:49.200: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.200: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container autoscaler ready: true, restart count 0 Dec 29 06:44:49.200: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.200: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:44:49.200: INFO: Container driver-registrar ready: true, restart count 0 Dec 29 06:44:49.200: INFO: Container mock ready: true, restart count 0 Dec 29 06:44:49.200: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.200: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:44:49.200: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:49.200: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:49.200: INFO: pod-2526fd38-277f-499e-b2c5-b55842c84c2d started at 2022-12-29 06:39:55 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:44:49.200: INFO: netserver-1 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.200: INFO: netserver-1 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.200: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container coredns ready: true, restart count 0 Dec 29 06:44:49.200: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:44:49.200: INFO: netserver-1 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.200: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.373: INFO: Latency metrics for node i-07228a57ea47d842b Dec 29 06:44:49.373: INFO: Logging node info for node i-07d5c844ec1aba791 Dec 29 06:44:49.403: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 23040 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volumemode-6468":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:44:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:44:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},},Config:nil,},} Dec 29 06:44:49.404: INFO: Logging kubelet events for node i-07d5c844ec1aba791 Dec 29 06:44:49.436: INFO: Logging pods the kubelet thinks is on node i-07d5c844ec1aba791 Dec 29 06:44:49.483: INFO: httpd started at 2022-12-29 06:42:01 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container httpd ready: false, restart count 0 Dec 29 06:44:49.483: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:44:32 +0000 UTC (0+7 container statuses recorded) Dec 29 06:44:49.483: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container csi-resizer ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container csi-snapshotter ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container hostpath ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container liveness-probe ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container node-driver-registrar ready: false, restart count 0 Dec 29 06:44:49.483: INFO: e2e-k9hf4-5ldkk started at 2022-12-29 06:44:45 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container c ready: false, restart count 0 Dec 29 06:44:49.483: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.483: INFO: Container jessie-querier ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container querier ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.483: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container csi-volume-tester ready: true, restart count 0 Dec 29 06:44:49.483: INFO: netserver-2 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.483: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.483: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.483: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:49.483: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:49.483: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:49.483: INFO: simpletest.rc-lg82l started at 2022-12-29 06:44:20 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container nginx ready: false, restart count 0 Dec 29 06:44:49.483: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container webserver ready: true, restart count 0 Dec 29 06:44:49.483: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:49.483: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:49.483: INFO: pod-projected-configmaps-03cdd563-f67f-480c-8bad-23ae8f2d256a started at 2022-12-29 06:42:28 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.483: INFO: Container createcm-volume-test ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container delcm-volume-test ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container updcm-volume-test ready: false, restart count 0 Dec 29 06:44:49.483: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.483: INFO: busybox-3b39c30d-31e2-49ad-81c2-e1e37a94c517 started at 2022-12-29 06:39:50 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container busybox ready: false, restart count 0 Dec 29 06:44:49.483: INFO: e2e-k9hf4-48rpj started at <nil> (0+0 container statuses recorded) Dec 29 06:44:49.483: INFO: failed-jobs-history-limit-27871602-87bxk started at 2022-12-29 06:42:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container c ready: false, restart count 0 Dec 29 06:44:49.483: INFO: pfpod started at 2022-12-29 06:44:38 +0000 UTC (0+2 container statuses recorded) Dec 29 06:44:49.483: INFO: Container portforwardtester ready: false, restart count 0 Dec 29 06:44:49.483: INFO: Container readiness ready: false, restart count 0 Dec 29 06:44:49.483: INFO: hostpath-symlink-prep-provisioning-3263 started at 2022-12-29 06:39:49 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container init-volume-provisioning-3263 ready: false, restart count 0 Dec 29 06:44:49.483: INFO: failed-jobs-history-limit-27871601-lg7kl started at 2022-12-29 06:41:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container c ready: false, restart count 0 Dec 29 06:44:49.483: INFO: failed-jobs-history-limit-27871603-sr5mg started at 2022-12-29 06:43:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container c ready: false, restart count 0 Dec 29 06:44:49.483: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.483: INFO: failed-jobs-history-limit-27871600-p8jnr started at 2022-12-29 06:40:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container c ready: false, restart count 0 Dec 29 06:44:49.483: INFO: hostpath-injector started at 2022-12-29 06:40:02 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container hostpath-injector ready: false, restart count 0 Dec 29 06:44:49.483: INFO: netserver-2 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.483: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.483: INFO: failed-jobs-history-limit-27871604-jqgj9 started at 2022-12-29 06:44:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container c ready: false, restart count 0 Dec 29 06:44:49.483: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.483: INFO: netserver-2 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.483: INFO: sample-webhook-deployment-865554f4d9-n7km6 started at 2022-12-29 06:41:01 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container sample-webhook ready: false, restart count 0 Dec 29 06:44:49.483: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.483: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.717: INFO: Latency metrics for node i-07d5c844ec1aba791 Dec 29 06:44:49.717: INFO: Logging node info for node i-0b6ab0d2cddb81acd Dec 29 06:44:49.747: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 22981 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7083":"i-0b6ab0d2cddb81acd","ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2022-12-29 06:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:44:49.747: INFO: Logging kubelet events for node i-0b6ab0d2cddb81acd Dec 29 06:44:49.781: INFO: Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd Dec 29 06:44:49.817: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:44:49.817: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.817: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.817: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:44:49.817: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:44:49.817: INFO: Container mock ready: false, restart count 0 Dec 29 06:44:49.817: INFO: netserver-3 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.817: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.817: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.817: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.817: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container donothing ready: false, restart count 0 Dec 29 06:44:49.817: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.817: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:44:49.817: INFO: Container driver-registrar ready: true, restart count 0 Dec 29 06:44:49.817: INFO: Container mock ready: true, restart count 0 Dec 29 06:44:49.817: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.817: INFO: netserver-3 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.817: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:44:49.817: INFO: netserver-3 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container webserver ready: false, restart count 0 Dec 29 06:44:49.817: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:44:49.817: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded) Dec 29 06:44:49.817: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:44:49.817: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:44:49.817: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded) Dec 29 06:44:49.817: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:44:49.817: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:44:49.817: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:44:49.992: INFO: Latency metrics for node i-0b6ab0d2cddb81acd [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "provisioning-7696" for this suite. �[38;5;243m12/29/22 06:44:49.992�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sfile\sas\ssubpath\s\[LinuxOnly\]$'
test/e2e/framework/pod/output/output.go:237
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0xc003aa5680?, {0x7606e1a?, 0x0?}, 0xc003aa5680, 0x0, {0xc0024efea8, 0x1, 0x1}, 0x0?)
test/e2e/framework/pod/output/output.go:237 +0x176
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutput(...)
test/e2e/framework/pod/output/output.go:214
k8s.io/kubernetes/test/e2e/storage/testsuites.TestBasicSubpathFile(0xc0010c33b0?, {0xc002f67950?, 0x11?}, 0xc003aa5680?, {0x75d0089?, 0x262a967?})
test/e2e/storage/testsuites/subpath.go:489 +0x12a
k8s.io/kubernetes/test/e2e/storage/testsuites.TestBasicSubpath(...)
test/e2e/storage/testsuites/subpath.go:480
k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func6()
test/e2e/storage/testsuites/subpath.go:237 +0x1d8
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:36:36.663�[0m
Dec 29 06:36:36.663: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename provisioning �[38;5;243m12/29/22 06:36:36.664�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:36:36.755�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:36:36.814�[0m
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
test/e2e/framework/metrics/init/init.go:31
[It] should support file as subpath [LinuxOnly]
test/e2e/storage/testsuites/subpath.go:230
Dec 29 06:36:36.872: INFO: Creating resource for dynamic PV
Dec 29 06:36:36.872: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi}
�[1mSTEP:�[0m creating a StorageClass provisioning-9217-e2e-scd8929 �[38;5;243m12/29/22 06:36:36.873�[0m
�[1mSTEP:�[0m creating a claim �[38;5;243m12/29/22 06:36:36.904�[0m
Dec 29 06:36:36.904: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
�[1mSTEP:�[0m Creating pod pod-subpath-test-dynamicpv-lx9g �[38;5;243m12/29/22 06:36:36.966�[0m
�[1mSTEP:�[0m Creating a pod to test atomic-volume-subpath �[38;5;243m12/29/22 06:36:36.966�[0m
Dec 29 06:36:36.999: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-lx9g" in namespace "provisioning-9217" to be "Succeeded or Failed"
Dec 29 06:36:37.029: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 29.78246ms
Dec 29 06:36:39.059: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2.059770443s
Dec 29 06:36:41.062: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4.062966103s
Dec 29 06:36:43.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 6.060180915s
Dec 29 06:36:45.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 8.060319592s
Dec 29 06:36:47.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 10.060202141s
Dec 29 06:36:49.059: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 12.059952665s
Dec 29 06:36:51.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 14.061206163s
Dec 29 06:36:53.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 16.06076197s
Dec 29 06:36:55.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 18.060134715s
Dec 29 06:36:57.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 20.061402102s
Dec 29 06:36:59.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 22.060287812s
Dec 29 06:37:01.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 24.06050998s
Dec 29 06:37:03.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 26.060122127s
Dec 29 06:37:05.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 28.060147279s
Dec 29 06:37:07.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 30.060141061s
Dec 29 06:37:09.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 32.060991566s
Dec 29 06:37:11.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 34.060345674s
Dec 29 06:37:13.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 36.060326572s
Dec 29 06:37:15.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 38.06066344s
Dec 29 06:37:17.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 40.061042259s
Dec 29 06:37:19.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 42.060043602s
Dec 29 06:37:21.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 44.060530542s
Dec 29 06:37:23.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 46.060816857s
Dec 29 06:37:25.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 48.060324416s
Dec 29 06:37:27.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 50.060029654s
Dec 29 06:37:29.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 52.060263662s
Dec 29 06:37:31.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 54.0603907s
Dec 29 06:37:33.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 56.060455102s
Dec 29 06:37:35.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 58.060662111s
Dec 29 06:37:37.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.06013094s
Dec 29 06:37:39.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.060263821s
Dec 29 06:37:41.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.060381315s
Dec 29 06:37:43.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.060770978s
Dec 29 06:37:45.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.060061417s
Dec 29 06:37:47.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.060498832s
Dec 29 06:37:49.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.060396619s
Dec 29 06:37:51.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.060266256s
Dec 29 06:37:53.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.060183995s
Dec 29 06:37:55.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.060773141s
Dec 29 06:37:57.059: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.059986493s
Dec 29 06:37:59.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.061044955s
Dec 29 06:38:01.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.061644648s
Dec 29 06:38:03.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.060810991s
Dec 29 06:38:05.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.060667154s
Dec 29 06:38:07.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.060984534s
Dec 29 06:38:09.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.060609873s
Dec 29 06:38:11.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.060475316s
Dec 29 06:38:13.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.060167766s
Dec 29 06:38:15.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.061020196s
Dec 29 06:38:17.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.060518685s
Dec 29 06:38:19.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.060965692s
Dec 29 06:38:21.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.060581084s
Dec 29 06:38:23.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.060069265s
Dec 29 06:38:25.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.060987112s
Dec 29 06:38:27.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.060820835s
Dec 29 06:38:29.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.061029869s
Dec 29 06:38:31.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.061247124s
Dec 29 06:38:33.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.060956764s
Dec 29 06:38:35.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.060265375s
Dec 29 06:38:37.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.060083404s
Dec 29 06:38:39.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.060513802s
Dec 29 06:38:41.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.060324425s
Dec 29 06:38:43.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.060105911s
Dec 29 06:38:45.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.060824205s
Dec 29 06:38:47.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.060480866s
Dec 29 06:38:49.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.061346327s
Dec 29 06:38:51.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.060425047s
Dec 29 06:38:53.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.060560566s
Dec 29 06:38:55.059: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.059951722s
Dec 29 06:38:57.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.060318902s
Dec 29 06:38:59.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.060450684s
Dec 29 06:39:01.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.060286643s
Dec 29 06:39:03.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.060177862s
Dec 29 06:39:05.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.060288022s
Dec 29 06:39:07.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.060710352s
Dec 29 06:39:09.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.060571755s
Dec 29 06:39:11.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.060964552s
Dec 29 06:39:13.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.061311447s
Dec 29 06:39:15.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.060692799s
Dec 29 06:39:17.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.060132781s
Dec 29 06:39:19.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.061190885s
Dec 29 06:39:21.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.061110331s
Dec 29 06:39:23.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.060690343s
Dec 29 06:39:25.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.060110061s
Dec 29 06:39:27.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.060568312s
Dec 29 06:39:29.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.060171372s
Dec 29 06:39:31.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.061862346s
Dec 29 06:39:33.059: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.059744772s
Dec 29 06:39:35.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.060941808s
Dec 29 06:39:37.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.061185457s
Dec 29 06:39:39.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.061199358s
Dec 29 06:39:41.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.060980792s
Dec 29 06:39:43.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.06011887s
Dec 29 06:39:45.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.060463917s
Dec 29 06:39:47.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.060903065s
Dec 29 06:39:49.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.061253689s
Dec 29 06:39:51.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.061197826s
Dec 29 06:39:53.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.061514651s
Dec 29 06:39:55.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.061236262s
Dec 29 06:39:57.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.060371542s
Dec 29 06:39:59.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.060132231s
Dec 29 06:40:01.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.061059341s
Dec 29 06:40:03.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.060404342s
Dec 29 06:40:05.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.060566947s
Dec 29 06:40:07.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.060246387s
Dec 29 06:40:09.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.060892374s
Dec 29 06:40:11.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.060302015s
Dec 29 06:40:13.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.060871849s
Dec 29 06:40:15.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.060614459s
Dec 29 06:40:17.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.06023473s
Dec 29 06:40:19.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.061322483s
Dec 29 06:40:21.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.060272933s
Dec 29 06:40:23.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.061033918s
Dec 29 06:40:25.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.060178302s
Dec 29 06:40:27.059: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.059966946s
Dec 29 06:40:29.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.060794274s
Dec 29 06:40:31.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.060260195s
Dec 29 06:40:33.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.060409863s
Dec 29 06:40:35.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.060433812s
Dec 29 06:40:37.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.060681338s
Dec 29 06:40:39.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.06139372s
Dec 29 06:40:41.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.060324465s
Dec 29 06:40:43.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.060192461s
Dec 29 06:40:45.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.060228786s
Dec 29 06:40:47.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.061232842s
Dec 29 06:40:49.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.061420535s
Dec 29 06:40:51.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.060762643s
Dec 29 06:40:53.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.060228137s
Dec 29 06:40:55.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.060780112s
Dec 29 06:40:57.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.060579112s
Dec 29 06:40:59.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.061177358s
Dec 29 06:41:01.062: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.062071796s
Dec 29 06:41:03.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.060041224s
Dec 29 06:41:05.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.061256835s
Dec 29 06:41:07.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.060390974s
Dec 29 06:41:09.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.060394801s
Dec 29 06:41:11.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.060433596s
Dec 29 06:41:13.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.061176378s
Dec 29 06:41:15.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.060329741s
Dec 29 06:41:17.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.060291387s
Dec 29 06:41:19.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.061370098s
Dec 29 06:41:21.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.060785841s
Dec 29 06:41:23.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.061370746s
Dec 29 06:41:25.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.060251639s
Dec 29 06:41:27.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.061547524s
Dec 29 06:41:29.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.061090717s
Dec 29 06:41:31.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.061165938s
Dec 29 06:41:33.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.060639351s
Dec 29 06:41:35.060: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.060468295s
Dec 29 06:41:37.061: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.061299019s
Dec 29 06:41:37.091: INFO: Pod "pod-subpath-test-dynamicpv-lx9g": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.091313898s
Dec 29 06:41:37.154: INFO: Failed to get logs from node "i-0345e6864820ace32" pod "pod-subpath-test-dynamicpv-lx9g" container "init-volume-dynamicpv-lx9g": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-lx9g)
Dec 29 06:41:37.185: INFO: Failed to get logs from node "i-0345e6864820ace32" pod "pod-subpath-test-dynamicpv-lx9g" container "test-container-subpath-dynamicpv-lx9g": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-lx9g)
�[1mSTEP:�[0m delete the pod �[38;5;243m12/29/22 06:41:37.185�[0m
Dec 29 06:41:37.220: INFO: Waiting for pod pod-subpath-test-dynamicpv-lx9g to disappear
Dec 29 06:41:37.250: INFO: Pod pod-subpath-test-dynamicpv-lx9g still exists
Dec 29 06:41:39.250: INFO: Waiting for pod pod-subpath-test-dynamicpv-lx9g to disappear
Dec 29 06:41:39.281: INFO: Pod pod-subpath-test-dynamicpv-lx9g no longer exists
Dec 29 06:41:39.281: INFO: Unexpected error:
<*errors.errorString | 0xc000606890>: {
s: "expected pod \"pod-subpath-test-dynamicpv-lx9g\" success: timed out while waiting for pod provisioning-9217/pod-subpath-test-dynamicpv-lx9g to be Succeeded or Failed",
}
Dec 29 06:41:39.281: FAIL: expected pod "pod-subpath-test-dynamicpv-lx9g" success: timed out while waiting for pod provisioning-9217/pod-subpath-test-dynamicpv-lx9g to be Succeeded or Failed
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0xc003aa5680?, {0x7606e1a?, 0x0?}, 0xc003aa5680, 0x0, {0xc0024efea8, 0x1, 0x1}, 0x0?)
test/e2e/framework/pod/output/output.go:237 +0x176
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutput(...)
test/e2e/framework/pod/output/output.go:214
k8s.io/kubernetes/test/e2e/storage/testsuites.TestBasicSubpathFile(0xc0010c33b0?, {0xc002f67950?, 0x11?}, 0xc003aa5680?, {0x75d0089?, 0x262a967?})
test/e2e/storage/testsuites/subpath.go:489 +0x12a
k8s.io/kubernetes/test/e2e/storage/testsuites.TestBasicSubpath(...)
test/e2e/storage/testsuites/subpath.go:480
k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func6()
test/e2e/storage/testsuites/subpath.go:237 +0x1d8
�[1mSTEP:�[0m Deleting pod �[38;5;243m12/29/22 06:41:39.281�[0m
Dec 29 06:41:39.281: INFO: Deleting pod "pod-subpath-test-dynamicpv-lx9g" in namespace "provisioning-9217"
�[1mSTEP:�[0m Deleting pvc �[38;5;243m12/29/22 06:41:39.311�[0m
Dec 29 06:41:39.370: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comkcstz"
Dec 29 06:41:39.402: INFO: Waiting up to 5m0s for PersistentVolume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279 to get deleted
Dec 29 06:41:39.431: INFO: PersistentVolume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279 found and phase=Released (29.606819ms)
Dec 29 06:41:44.463: INFO: PersistentVolume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279 found and phase=Released (5.061083597s)
Dec 29 06:41:49.494: INFO: PersistentVolume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279 found and phase=Released (10.091807009s)
Dec 29 06:41:54.525: INFO: PersistentVolume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279 found and phase=Released (15.122892321s)
Dec 29 06:41:59.556: INFO: PersistentVolume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279 was removed
�[1mSTEP:�[0m Deleting sc �[38;5;243m12/29/22 06:41:59.556�[0m
[AfterEach] [Testpattern: Dynamic PV (default fs)] subPath
test/e2e/framework/node/init/init.go:32
Dec 29 06:41:59.589: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:41:59.62�[0m
�[1mSTEP:�[0m Collecting events from namespace "provisioning-9217". �[38;5;243m12/29/22 06:41:59.62�[0m
�[1mSTEP:�[0m Found 16 events. �[38;5;243m12/29/22 06:41:59.651�[0m
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:36 +0000 UTC - event for ebs.csi.aws.comkcstz: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:36 +0000 UTC - event for ebs.csi.aws.comkcstz: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:36 +0000 UTC - event for ebs.csi.aws.comkcstz: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-j8vxm_47cfaf09-8c13-4f52-9a11-ea60ac01cfa3 } Provisioning: External provisioner is provisioning volume for claim "provisioning-9217/ebs.csi.aws.comkcstz"
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:40 +0000 UTC - event for ebs.csi.aws.comkcstz: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-j8vxm_47cfaf09-8c13-4f52-9a11-ea60ac01cfa3 } ProvisioningSucceeded: Successfully provisioned volume pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:40 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {default-scheduler } Scheduled: Successfully assigned provisioning-9217/pod-subpath-test-dynamicpv-lx9g to i-0345e6864820ace32
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:43 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-226e9e4b-4156-4c12-850e-2ea0ddc12279"
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:48 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "77c90bc49eb590cc922d70ef1d6fb0d0cc606b6d8f95dc5ab12ee65e70a7a6b6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:36:59 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "659e46ee5527b593bc1627a0597ea9b1b791566503ce76324faa67d175aa2c6c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:37:12 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e04ea43542832b0f4e6744a44c3da665c20bc94d8ccd6857e58d098b03d2d90a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:37:27 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "aa3e48f33daeb3d1b25d41bee744b141e159461b648c7084f8ea612f044eb3b8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:37:39 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "119b8083ad60b28bc1bb3b7ba3a42d0eec3e0feb199629a2577833ac3ddbda01": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:37:52 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "34b4e4518e3e5bb5fceb9c01fb85135e6c217b18efac364958597537c1990455": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:38:07 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0ae7a55c9ca2eb864aff9e54d8af7f41d06e6841a76dfc7d56b4e20f0e5508eb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:38:21 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d216f079f182c4aa18896d0c91a94b158322868a67d2a1622d810bbcd4ca2e4d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:38:36 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "70b02faa4e9fd008af7ae67d27a5f3a87b924114ff7667a0bc398b59f45677da": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.651: INFO: At 2022-12-29 06:38:48 +0000 UTC - event for pod-subpath-test-dynamicpv-lx9g: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4e2f48921300c383857772370ff1f3d3ca2157ec45aa7ed8181be9472054b74b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:41:59.681: INFO: POD NODE PHASE GRACE CONDITIONS
Dec 29 06:41:59.681: INFO:
Dec 29 06:41:59.712: INFO:
Logging node info for node i-0345e6864820ace32
Dec 29 06:41:59.742: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 21638 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:41:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:41 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:41 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:41 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:41 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:41:59.742: INFO:
Logging kubelet events for node i-0345e6864820ace32
Dec 29 06:41:59.779: INFO:
Logging pods the kubelet thinks is on node i-0345e6864820ace32
Dec 29 06:41:59.815: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:41:59.815: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:41:59.815: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container webserver ready: false, restart count 0
Dec 29 06:41:59.815: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:41:59.815: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:41:59.815: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: pod-315df80e-0f06-4d38-875a-df43417f95d3 started at 2022-12-29 06:37:23 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container test-container ready: false, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dwftm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvtcq started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container coredns ready: true, restart count 0
Dec 29 06:41:59.815: INFO: pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 started at 2022-12-29 06:37:26 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container nginx ready: false, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gtk84 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: inline-volume-tester2-lkk7j started at 2022-12-29 06:41:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container csi-volume-tester ready: false, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-ck56n started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.815: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-mcnp2 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:41:59.815: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:41:59.982: INFO:
Latency metrics for node i-0345e6864820ace32
Dec 29 06:41:59.982: INFO:
Logging node info for node i-06be1dd8c97acf782
Dec 29 06:42:00.012: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 21540 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:41:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:42:00.013: INFO:
Logging kubelet events for node i-06be1dd8c97acf782
Dec 29 06:42:00.045: INFO:
Logging pods the kubelet thinks is on node i-06be1dd8c97acf782
Dec 29 06:42:00.081: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.081: INFO: Container kube-controller-manager ready: true, restart count 2
Dec 29 06:42:00.081: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:42:00.081: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:42:00.081: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.081: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:42:00.081: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.081: INFO: Container kops-controller ready: true, restart count 0
Dec 29 06:42:00.081: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded)
Dec 29 06:42:00.081: INFO: Container csi-attacher ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:42:00.081: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.081: INFO: Container etcd-manager ready: true, restart count 0
Dec 29 06:42:00.081: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded)
Dec 29 06:42:00.081: INFO: Container healthcheck ready: true, restart count 0
Dec 29 06:42:00.081: INFO: Container kube-apiserver ready: true, restart count 1
Dec 29 06:42:00.081: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.082: INFO: Container dns-controller ready: true, restart count 0
Dec 29 06:42:00.082: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.082: INFO: Container cilium-operator ready: true, restart count 0
Dec 29 06:42:00.082: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.082: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Dec 29 06:42:00.082: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.082: INFO: Container etcd-manager ready: true, restart count 0
Dec 29 06:42:00.082: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.082: INFO: Container kube-scheduler ready: true, restart count 0
Dec 29 06:42:00.243: INFO:
Latency metrics for node i-06be1dd8c97acf782
Dec 29 06:42:00.243: INFO:
Logging node info for node i-07228a57ea47d842b
Dec 29 06:42:00.275: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 20587 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9651^9428e159-8743-11ed-b070-a6882e2b2850],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:42:00.275: INFO:
Logging kubelet events for node i-07228a57ea47d842b
Dec 29 06:42:00.308: INFO:
Logging pods the kubelet thinks is on node i-07228a57ea47d842b
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-57b2m started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:42:00.349: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container webserver ready: false, restart count 0
Dec 29 06:42:00.349: INFO: hostexec-i-07228a57ea47d842b-9fc7b started at 2022-12-29 06:39:25 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:42:00.349: INFO: pod-592477cc-870d-47db-b10f-254f00e2207c started at 2022-12-29 06:39:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container write-pod ready: false, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:42:00.349: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:42:00.349: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b6wc8 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: pod-2526fd38-277f-499e-b2c5-b55842c84c2d started at 2022-12-29 06:39:55 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container write-pod ready: false, restart count 0
Dec 29 06:42:00.349: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container coredns ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b9h6m started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4xmqc started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container volume-tester ready: false, restart count 0
Dec 29 06:42:00.349: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:42:00.349: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:42:00.349: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4l6sw started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.349: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container autoscaler ready: true, restart count 0
Dec 29 06:42:00.349: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.349: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:42:00.349: INFO: Container driver-registrar ready: true, restart count 0
Dec 29 06:42:00.349: INFO: Container mock ready: true, restart count 0
Dec 29 06:42:00.552: INFO:
Latency metrics for node i-07228a57ea47d842b
Dec 29 06:42:00.552: INFO:
Logging node info for node i-07d5c844ec1aba791
Dec 29 06:42:00.584: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 21778 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6856":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:41:58 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:41:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf,DevicePath:,},},Config:nil,},}
Dec 29 06:42:00.584: INFO:
Logging kubelet events for node i-07d5c844ec1aba791
Dec 29 06:42:00.617: INFO:
Logging pods the kubelet thinks is on node i-07d5c844ec1aba791
Dec 29 06:42:00.656: INFO: pod-0 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container donothing ready: true, restart count 0
Dec 29 06:42:00.656: INFO: sample-webhook-deployment-865554f4d9-n7km6 started at 2022-12-29 06:41:01 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container sample-webhook ready: false, restart count 0
Dec 29 06:42:00.656: INFO: pod-1 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container donothing ready: false, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zlzqg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:42:00.656: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container jessie-querier ready: false, restart count 0
Dec 29 06:42:00.656: INFO: Container querier ready: false, restart count 0
Dec 29 06:42:00.656: INFO: Container webserver ready: false, restart count 0
Dec 29 06:42:00.656: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container csi-volume-tester ready: true, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9v6vz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:42:00.656: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container webserver ready: true, restart count 0
Dec 29 06:42:00.656: INFO: pod-configmaps-191e839a-8743-4828-8230-52b8b3ce9a18 started at 2022-12-29 06:39:33 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container agnhost-container ready: false, restart count 0
Dec 29 06:42:00.656: INFO: pod-73af6e22-e126-427f-bec8-33101a9f6c91 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container write-pod ready: false, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: busybox-3b39c30d-31e2-49ad-81c2-e1e37a94c517 started at 2022-12-29 06:39:50 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container busybox ready: false, restart count 0
Dec 29 06:42:00.656: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:33:49 +0000 UTC (0+7 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container csi-attacher ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container csi-snapshotter ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container hostpath ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:42:00.656: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:42:00.656: INFO: failed-jobs-history-limit-27871602-87bxk started at 2022-12-29 06:42:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container c ready: false, restart count 0
Dec 29 06:42:00.656: INFO: hostpath-symlink-prep-provisioning-3263 started at 2022-12-29 06:39:49 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container init-volume-provisioning-3263 ready: false, restart count 0
Dec 29 06:42:00.656: INFO: failed-jobs-history-limit-27871601-lg7kl started at 2022-12-29 06:41:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container c ready: false, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: failed-jobs-history-limit-27871600-p8jnr started at 2022-12-29 06:40:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container c ready: false, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-j7svz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:42:00.656: INFO: hostpath-injector started at 2022-12-29 06:40:02 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container hostpath-injector ready: false, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.656: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.656: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.851: INFO:
Latency metrics for node i-07d5c844ec1aba791
Dec 29 06:42:00.851: INFO:
Logging node info for node i-0b6ab0d2cddb81acd
Dec 29 06:42:00.881: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 19970 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:39:32 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420 kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420,DevicePath:,},},Config:nil,},}
Dec 29 06:42:00.882: INFO:
Logging kubelet events for node i-0b6ab0d2cddb81acd
Dec 29 06:42:00.914: INFO:
Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6h6vh started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 started at 2022-12-29 06:33:56 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container write-pod ready: false, restart count 0
Dec 29 06:42:00.954: INFO: hostexec-i-0b6ab0d2cddb81acd-qppr8 started at 2022-12-29 06:34:30 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:42:00.954: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:42:00.954: INFO: Container driver-registrar ready: false, restart count 0
Dec 29 06:42:00.954: INFO: Container mock ready: false, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container donothing ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: pod-subpath-test-dynamicpv-t84v started at 2022-12-29 06:39:29 +0000 UTC (0+2 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container test-container-subpath-dynamicpv-t84v ready: false, restart count 0
Dec 29 06:42:00.954: INFO: Container test-container-volume-dynamicpv-t84v ready: false, restart count 0
Dec 29 06:42:00.954: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:42:00.954: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:42:00.954: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:42:00.954: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:42:00.954: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:42:00.954: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:42:00.954: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d8lgx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-8qfzg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container webserver ready: false, restart count 0
Dec 29 06:42:00.954: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:42:00.954: INFO: Container driver-registrar ready: false, restart count 0
Dec 29 06:42:00.954: INFO: Container mock ready: false, restart count 0
Dec 29 06:42:00.954: INFO: local-client started at 2022-12-29 06:39:39 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container local-client ready: false, restart count 0
Dec 29 06:42:00.954: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:42:00.954: INFO: downwardapi-volume-cc6aea7e-266b-4611-a490-6cb3559eaf08 started at 2022-12-29 06:39:43 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:42:00.954: INFO: Container client-container ready: false, restart count 0
Dec 29 06:42:01.170: INFO:
Latency metrics for node i-0b6ab0d2cddb81acd
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] subPath
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "provisioning-9217" for this suite. �[38;5;243m12/29/22 06:42:01.17�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sGenerated\sclientset\sshould\screate\spods\,\sset\sthe\sdeletionTimestamp\sand\sdeletionGracePeriodSeconds\sof\sthe\spod$'
test/e2e/apimachinery/generated_clientset.go:151 k8s.io/kubernetes/test/e2e/apimachinery.glob..func14.1() test/e2e/apimachinery/generated_clientset.go:151 +0x6b5from junit_01.xml
[BeforeEach] [sig-api-machinery] Generated clientset set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:37:26.462�[0m Dec 29 06:37:26.462: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename clientset �[38;5;243m12/29/22 06:37:26.463�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:37:26.554�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:37:26.612�[0m [BeforeEach] [sig-api-machinery] Generated clientset test/e2e/framework/metrics/init/init.go:31 [It] should create pods, set the deletionTimestamp and deletionGracePeriodSeconds of the pod test/e2e/apimachinery/generated_clientset.go:105 �[1mSTEP:�[0m constructing the pod �[38;5;243m12/29/22 06:37:26.67�[0m �[1mSTEP:�[0m setting up watch �[38;5;243m12/29/22 06:37:26.67�[0m �[1mSTEP:�[0m creating the pod �[38;5;243m12/29/22 06:37:26.729�[0m �[1mSTEP:�[0m verifying the pod is in kubernetes �[38;5;243m12/29/22 06:37:26.761�[0m �[1mSTEP:�[0m verifying pod creation was observed �[38;5;243m12/29/22 06:37:26.79�[0m Dec 29 06:37:26.791: INFO: Waiting up to 5m0s for pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8" in namespace "clientset-4329" to be "running" Dec 29 06:37:26.820: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 29.645232ms Dec 29 06:37:28.853: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.062123156s Dec 29 06:37:30.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4.060112911s Dec 29 06:37:32.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 6.060815036s Dec 29 06:37:34.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 8.061231929s Dec 29 06:37:36.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 10.060210454s Dec 29 06:37:38.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 12.060126805s Dec 29 06:37:40.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 14.060290282s Dec 29 06:37:42.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 16.05996303s Dec 29 06:37:44.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 18.060257531s Dec 29 06:37:46.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 20.060259165s Dec 29 06:37:48.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 22.060582948s Dec 29 06:37:50.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 24.060760394s Dec 29 06:37:52.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 26.059969363s Dec 29 06:37:54.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 28.060470937s Dec 29 06:37:56.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 30.060486922s Dec 29 06:37:58.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 32.059923954s Dec 29 06:38:00.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 34.059829882s Dec 29 06:38:02.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 36.05958014s Dec 29 06:38:04.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 38.060546906s Dec 29 06:38:06.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 40.060090492s Dec 29 06:38:08.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 42.060513818s Dec 29 06:38:10.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 44.060776902s Dec 29 06:38:12.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 46.060214462s Dec 29 06:38:14.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 48.060318454s Dec 29 06:38:16.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 50.059729976s Dec 29 06:38:18.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 52.059934012s Dec 29 06:38:20.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 54.059973539s Dec 29 06:38:22.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 56.060037889s Dec 29 06:38:24.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 58.060764444s Dec 29 06:38:26.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.060437162s Dec 29 06:38:28.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.0605418s Dec 29 06:38:30.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.060347558s Dec 29 06:38:32.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.060073882s Dec 29 06:38:34.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.060155324s Dec 29 06:38:36.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.060040099s Dec 29 06:38:38.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.060206752s Dec 29 06:38:40.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.060819093s Dec 29 06:38:42.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.059905941s Dec 29 06:38:44.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.06115851s Dec 29 06:38:46.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.059822842s Dec 29 06:38:48.853: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.062553294s Dec 29 06:38:50.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.0602007s Dec 29 06:38:52.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.060274198s Dec 29 06:38:54.854: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.063695485s Dec 29 06:38:56.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.059924611s Dec 29 06:38:58.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.06049232s Dec 29 06:39:00.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.060412205s Dec 29 06:39:02.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.060997784s Dec 29 06:39:04.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.059840825s Dec 29 06:39:06.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.060085088s Dec 29 06:39:08.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.060418584s Dec 29 06:39:10.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.060002462s Dec 29 06:39:12.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.060001669s Dec 29 06:39:14.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.060168969s Dec 29 06:39:16.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.06013833s Dec 29 06:39:18.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.059963792s Dec 29 06:39:20.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.06063412s Dec 29 06:39:22.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.059838012s Dec 29 06:39:24.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.060995209s Dec 29 06:39:26.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.061593136s Dec 29 06:39:28.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.061549093s Dec 29 06:39:30.854: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.063466238s Dec 29 06:39:32.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.060806792s Dec 29 06:39:34.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.05983227s Dec 29 06:39:36.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.06029322s Dec 29 06:39:38.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.06007505s Dec 29 06:39:40.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.060533853s Dec 29 06:39:42.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.06040097s Dec 29 06:39:44.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.05984909s Dec 29 06:39:46.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.059990226s Dec 29 06:39:48.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.059743175s Dec 29 06:39:50.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.06051888s Dec 29 06:39:52.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.059852065s Dec 29 06:39:54.859: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.068405819s Dec 29 06:39:56.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.060264236s Dec 29 06:39:58.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.059860734s Dec 29 06:40:00.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.060293492s Dec 29 06:40:02.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.060550936s Dec 29 06:40:04.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.060428529s Dec 29 06:40:06.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.059919161s Dec 29 06:40:08.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.060179538s Dec 29 06:40:10.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.060486729s Dec 29 06:40:12.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.060841619s Dec 29 06:40:14.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.06006999s Dec 29 06:40:16.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.060352846s Dec 29 06:40:18.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.06006395s Dec 29 06:40:20.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.059671687s Dec 29 06:40:22.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.060513768s Dec 29 06:40:24.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.060642191s Dec 29 06:40:26.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.060040032s Dec 29 06:40:28.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.060084914s Dec 29 06:40:30.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.060799179s Dec 29 06:40:32.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.059588063s Dec 29 06:40:34.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.059969726s Dec 29 06:40:36.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.060363778s Dec 29 06:40:38.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.060406641s Dec 29 06:40:40.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.060229091s Dec 29 06:40:42.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.060093722s Dec 29 06:40:44.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.059915744s Dec 29 06:40:46.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.060058677s Dec 29 06:40:48.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.059912594s Dec 29 06:40:50.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.060461677s Dec 29 06:40:52.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.059911484s Dec 29 06:40:54.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.059712542s Dec 29 06:40:56.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.059604186s Dec 29 06:40:58.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.059956663s Dec 29 06:41:00.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.061362124s Dec 29 06:41:02.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.060000606s Dec 29 06:41:04.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.059996005s Dec 29 06:41:06.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.060360558s Dec 29 06:41:08.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.060291887s Dec 29 06:41:10.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.060409271s Dec 29 06:41:12.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.059613513s Dec 29 06:41:14.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.060399617s Dec 29 06:41:16.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.060438418s Dec 29 06:41:18.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.060182732s Dec 29 06:41:20.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.060174098s Dec 29 06:41:22.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.059920354s Dec 29 06:41:24.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.060254651s Dec 29 06:41:26.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.060296408s Dec 29 06:41:28.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.059951809s Dec 29 06:41:30.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.059674921s Dec 29 06:41:32.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.06075939s Dec 29 06:41:34.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.060342727s Dec 29 06:41:36.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.060498173s Dec 29 06:41:38.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.060284889s Dec 29 06:41:40.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.059952352s Dec 29 06:41:42.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.060181036s Dec 29 06:41:44.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.060973324s Dec 29 06:41:46.852: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.061176753s Dec 29 06:41:48.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.060905337s Dec 29 06:41:50.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.059756728s Dec 29 06:41:52.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.060626619s Dec 29 06:41:54.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.060137927s Dec 29 06:41:56.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.06019415s Dec 29 06:41:58.855: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.064266528s Dec 29 06:42:00.853: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.062662573s Dec 29 06:42:02.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.060196034s Dec 29 06:42:04.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.06026457s Dec 29 06:42:06.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.060488046s Dec 29 06:42:08.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.060483119s Dec 29 06:42:10.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.059604081s Dec 29 06:42:12.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.060198036s Dec 29 06:42:14.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.059787765s Dec 29 06:42:16.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.0608115s Dec 29 06:42:18.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.060744274s Dec 29 06:42:20.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.060084625s Dec 29 06:42:22.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.060374962s Dec 29 06:42:24.851: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.060251406s Dec 29 06:42:26.850: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.05984868s Dec 29 06:42:26.880: INFO: Pod "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.089362852s Dec 29 06:42:26.881: INFO: Unexpected error: <*pod.timeoutError | 0xc0006b8a20>: { msg: "timed out while waiting for pod clientset-4329/pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 to be running", observedObjects: [ <*v1.Pod | 0xc003fb2900>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8", GenerateName: "", Namespace: "clientset-4329", SelfLink: "", UID: "10ae76fb-9e2a-4d22-a9b9-7bc1b871d596", ResourceVersion: "19056", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63807892646, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: {"time": "670981268", "name": "foo"}, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63807892646, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:name\":{},\"f:time\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"nginx\\\"}\":{\".\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":80,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:protocol\":{}}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63807892649, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "kube-api-access-cmwdx", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, { Secret: ..., Downw... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 29 06:42:26.881: FAIL: timed out while waiting for pod clientset-4329/pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 to be running Full Stack Trace k8s.io/kubernetes/test/e2e/apimachinery.glob..func14.1() test/e2e/apimachinery/generated_clientset.go:151 +0x6b5 [AfterEach] [sig-api-machinery] Generated clientset test/e2e/framework/node/init/init.go:32 Dec 29 06:42:26.881: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] Generated clientset test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-api-machinery] Generated clientset dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:42:26.912�[0m �[1mSTEP:�[0m Collecting events from namespace "clientset-4329". �[38;5;243m12/29/22 06:42:26.912�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m12/29/22 06:42:26.942�[0m Dec 29 06:42:26.942: INFO: At 2022-12-29 06:37:26 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {default-scheduler } Scheduled: Successfully assigned clientset-4329/pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 to i-0345e6864820ace32 Dec 29 06:42:26.942: INFO: At 2022-12-29 06:37:27 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a7c5976f499d5e5f737cf22a88ec29583d3c6b8d0a600a39b24167e9dfbc7ae3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:37:43 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fde7caa06cca34f49d9425a1144515b19ab6b251f764023671ffbb3c6e861296": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:37:58 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cf02843bc03c5caa0ce5e4791b7af97af77bb48365ae183d5dc101242c3adf41": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:38:10 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "712b1fe890d29fd6c17bdd7f3dfbf178e2d21cb39a95468a8bdd547aed552d6e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:38:21 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b6e2e8b25791d39194f2d28eee81840ebdb0758c12b918f015580623c1d90ec4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:38:35 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1ef2d50fae52275d572345e3e56d762d9f936836431ed36331d33ea10bccb315": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:38:50 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7e346638b1e3c57ee1d572ba431e43736c878a2096dfb1e43bf40f2b8f684a01": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:39:04 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "46377a648a53c719bc5488bf5ae92c300463b9600f63ac3c4c9501025f0444aa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:39:18 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7b910b8e9f2f93c71147d7d829f55e06902825850c585ac7ec8ac9546acec1d9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.942: INFO: At 2022-12-29 06:39:32 +0000 UTC - event for pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8: {kubelet i-0345e6864820ace32} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "17d281a6637866a8f52c7ed4036b87bbfa188bbf41fe054a2f9c9ccff2d38bff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:42:26.972: INFO: POD NODE PHASE GRACE CONDITIONS Dec 29 06:42:26.972: INFO: pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 i-0345e6864820ace32 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:37:26 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:37:26 +0000 UTC ContainersNotReady containers with unready status: [nginx]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:37:26 +0000 UTC ContainersNotReady containers with unready status: [nginx]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:37:26 +0000 UTC }] Dec 29 06:42:26.972: INFO: Dec 29 06:42:27.003: INFO: Unable to fetch clientset-4329/pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8/nginx logs: the server rejected our request for an unknown reason (get pods pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8) Dec 29 06:42:27.034: INFO: Logging node info for node i-0345e6864820ace32 Dec 29 06:42:27.064: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 21806 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799,DevicePath:,},},Config:nil,},} Dec 29 06:42:27.064: INFO: Logging kubelet events for node i-0345e6864820ace32 Dec 29 06:42:27.096: INFO: Logging pods the kubelet thinks is on node i-0345e6864820ace32 Dec 29 06:42:27.132: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container coredns ready: true, restart count 0 Dec 29 06:42:27.132: INFO: pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 started at 2022-12-29 06:37:26 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container nginx ready: false, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: netserver-0 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gtk84 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: inline-volume-tester2-lkk7j started at 2022-12-29 06:41:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container csi-volume-tester ready: false, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-ck56n started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-mcnp2 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:42:27.132: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:42:27.132: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container webserver ready: true, restart count 0 Dec 29 06:42:27.132: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:27.132: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:42:27.132: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:27.132: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dwftm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.132: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvtcq started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.132: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.328: INFO: Latency metrics for node i-0345e6864820ace32 Dec 29 06:42:27.328: INFO: Logging node info for node i-06be1dd8c97acf782 Dec 29 06:42:27.358: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 21540 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:41:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:42:27.358: INFO: Logging kubelet events for node i-06be1dd8c97acf782 Dec 29 06:42:27.390: INFO: Logging pods the kubelet thinks is on node i-06be1dd8c97acf782 Dec 29 06:42:27.427: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:42:27.427: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container kube-scheduler ready: true, restart count 0 Dec 29 06:42:27.427: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container dns-controller ready: true, restart count 0 Dec 29 06:42:27.427: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container cilium-operator ready: true, restart count 0 Dec 29 06:42:27.427: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Dec 29 06:42:27.427: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container kops-controller ready: true, restart count 0 Dec 29 06:42:27.427: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded) Dec 29 06:42:27.427: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:27.427: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:42:27.427: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded) Dec 29 06:42:27.427: INFO: Container healthcheck ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container kube-apiserver ready: true, restart count 1 Dec 29 06:42:27.427: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Container kube-controller-manager ready: true, restart count 2 Dec 29 06:42:27.427: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded) Dec 29 06:42:27.427: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:42:27.427: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:27.427: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:27.427: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:42:27.582: INFO: Latency metrics for node i-06be1dd8c97acf782 Dec 29 06:42:27.582: INFO: Logging node info for node i-07228a57ea47d842b Dec 29 06:42:27.612: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 20587 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9651^9428e159-8743-11ed-b070-a6882e2b2850],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:42:27.612: INFO: Logging kubelet events for node i-07228a57ea47d842b Dec 29 06:42:27.644: INFO: Logging pods the kubelet thinks is on node i-07228a57ea47d842b Dec 29 06:42:27.682: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:27.682: INFO: hostexec-i-07228a57ea47d842b-9fc7b started at 2022-12-29 06:39:25 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-57b2m started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:42:27.682: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b6wc8 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: pod-592477cc-870d-47db-b10f-254f00e2207c started at 2022-12-29 06:39:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: pod-2526fd38-277f-499e-b2c5-b55842c84c2d started at 2022-12-29 06:39:55 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:42:27.682: INFO: netserver-1 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:27.682: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container coredns ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b9h6m started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4xmqc started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:27.682: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:42:27.682: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:27.682: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container volume-tester ready: false, restart count 0 Dec 29 06:42:27.682: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container autoscaler ready: true, restart count 0 Dec 29 06:42:27.682: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:27.682: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:42:27.682: INFO: Container driver-registrar ready: true, restart count 0 Dec 29 06:42:27.682: INFO: Container mock ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4l6sw started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.682: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.682: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.871: INFO: Latency metrics for node i-07228a57ea47d842b Dec 29 06:42:27.871: INFO: Logging node info for node i-07d5c844ec1aba791 Dec 29 06:42:27.901: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 21778 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6856":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:41:58 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:41:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:58 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf,DevicePath:,},},Config:nil,},} Dec 29 06:42:27.902: INFO: Logging kubelet events for node i-07d5c844ec1aba791 Dec 29 06:42:27.934: INFO: Logging pods the kubelet thinks is on node i-07d5c844ec1aba791 Dec 29 06:42:27.978: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container webserver ready: true, restart count 0 Dec 29 06:42:27.978: INFO: pod-configmaps-191e839a-8743-4828-8230-52b8b3ce9a18 started at 2022-12-29 06:39:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container agnhost-container ready: false, restart count 0 Dec 29 06:42:27.978: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: busybox-3b39c30d-31e2-49ad-81c2-e1e37a94c517 started at 2022-12-29 06:39:50 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container busybox ready: false, restart count 0 Dec 29 06:42:27.978: INFO: pod-73af6e22-e126-427f-bec8-33101a9f6c91 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:33:49 +0000 UTC (0+7 container statuses recorded) Dec 29 06:42:27.978: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container csi-snapshotter ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container hostpath ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:42:27.978: INFO: failed-jobs-history-limit-27871602-87bxk started at 2022-12-29 06:42:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container c ready: false, restart count 0 Dec 29 06:42:27.978: INFO: hostpath-symlink-prep-provisioning-3263 started at 2022-12-29 06:39:49 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container init-volume-provisioning-3263 ready: false, restart count 0 Dec 29 06:42:27.978: INFO: failed-jobs-history-limit-27871601-lg7kl started at 2022-12-29 06:41:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container c ready: false, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: hostpath-injector started at 2022-12-29 06:40:02 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container hostpath-injector ready: false, restart count 0 Dec 29 06:42:27.978: INFO: failed-jobs-history-limit-27871600-p8jnr started at 2022-12-29 06:40:00 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container c ready: false, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-j7svz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: pod-0 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container donothing ready: true, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zlzqg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:42:27.978: INFO: netserver-2 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:27.978: INFO: sample-webhook-deployment-865554f4d9-n7km6 started at 2022-12-29 06:41:01 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container sample-webhook ready: false, restart count 0 Dec 29 06:42:27.978: INFO: pod-1 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container donothing ready: false, restart count 0 Dec 29 06:42:27.978: INFO: httpd started at 2022-12-29 06:42:01 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container httpd ready: false, restart count 0 Dec 29 06:42:27.978: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:27.978: INFO: Container jessie-querier ready: false, restart count 0 Dec 29 06:42:27.978: INFO: Container querier ready: false, restart count 0 Dec 29 06:42:27.978: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:27.978: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container csi-volume-tester ready: true, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9v6vz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:27.978: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:27.978: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:27.978: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:27.978: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:42:28.178: INFO: Latency metrics for node i-07d5c844ec1aba791 Dec 29 06:42:28.178: INFO: Logging node info for node i-0b6ab0d2cddb81acd Dec 29 06:42:28.208: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 19970 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:39:32 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420 kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420,DevicePath:,},},Config:nil,},} Dec 29 06:42:28.208: INFO: Logging kubelet events for node i-0b6ab0d2cddb81acd Dec 29 06:42:28.240: INFO: Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container donothing ready: true, restart count 0 Dec 29 06:42:28.279: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:28.279: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:42:28.279: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:42:28.279: INFO: Container mock ready: false, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: pod-subpath-test-dynamicpv-t84v started at 2022-12-29 06:39:29 +0000 UTC (0+2 container statuses recorded) Dec 29 06:42:28.279: INFO: Container test-container-subpath-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:42:28.279: INFO: Container test-container-volume-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:42:28.279: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:42:28.279: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:42:28.279: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:42:28.279: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:28.279: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:42:28.279: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:42:28.279: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:42:28.279: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d8lgx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-8qfzg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:28.279: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded) Dec 29 06:42:28.279: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:42:28.279: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:42:28.279: INFO: Container mock ready: false, restart count 0 Dec 29 06:42:28.279: INFO: local-client started at 2022-12-29 06:39:39 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container local-client ready: false, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: netserver-3 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container webserver ready: false, restart count 0 Dec 29 06:42:28.279: INFO: downwardapi-volume-cc6aea7e-266b-4611-a490-6cb3559eaf08 started at 2022-12-29 06:39:43 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container client-container ready: false, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6h6vh started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:42:28.279: INFO: pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 started at 2022-12-29 06:33:56 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:42:28.279: INFO: hostexec-i-0b6ab0d2cddb81acd-qppr8 started at 2022-12-29 06:34:30 +0000 UTC (0+1 container statuses recorded) Dec 29 06:42:28.279: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:42:28.446: INFO: Latency metrics for node i-0b6ab0d2cddb81acd [DeferCleanup (Each)] [sig-api-machinery] Generated clientset tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "clientset-4329" for this suite. �[38;5;243m12/29/22 06:42:28.447�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sCronJob\sshould\sdelete\sfailed\sfinished\sjobs\swith\slimit\sof\sone\sjob$'
test/e2e/apps/cronjob.go:488
k8s.io/kubernetes/test/e2e/apps.ensureHistoryLimits({0x801e128, 0xc003fd36c0}, {0xc0049126a0, 0xc}, 0x75cbe8b?)
test/e2e/apps/cronjob.go:488 +0x1dd
k8s.io/kubernetes/test/e2e/apps.glob..func2.9()
test/e2e/apps/cronjob.go:298 +0xf6
[BeforeEach] [sig-apps] CronJob
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:39:45.864�[0m
Dec 29 06:39:45.864: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename cronjob �[38;5;243m12/29/22 06:39:45.865�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:39:45.955�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:39:46.013�[0m
[BeforeEach] [sig-apps] CronJob
test/e2e/framework/metrics/init/init.go:31
[It] should delete failed finished jobs with limit of one job
test/e2e/apps/cronjob.go:291
�[1mSTEP:�[0m Creating an AllowConcurrent cronjob with custom history limit �[38;5;243m12/29/22 06:39:46.071�[0m
�[1mSTEP:�[0m Ensuring a finished job exists �[38;5;243m12/29/22 06:39:46.103�[0m
Dec 29 06:44:46.187: INFO: Unexpected error: Failed to ensure a finished cronjob exists in namespace cronjob-8627:
<*errors.errorString | 0xc0001efa20>: {
s: "timed out waiting for the condition",
}
Dec 29 06:44:46.188: FAIL: Failed to ensure a finished cronjob exists in namespace cronjob-8627: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/apps.ensureHistoryLimits({0x801e128, 0xc003fd36c0}, {0xc0049126a0, 0xc}, 0x75cbe8b?)
test/e2e/apps/cronjob.go:488 +0x1dd
k8s.io/kubernetes/test/e2e/apps.glob..func2.9()
test/e2e/apps/cronjob.go:298 +0xf6
[AfterEach] [sig-apps] CronJob
test/e2e/framework/node/init/init.go:32
Dec 29 06:44:46.188: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] CronJob
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] CronJob
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:44:46.221�[0m
�[1mSTEP:�[0m Collecting events from namespace "cronjob-8627". �[38;5;243m12/29/22 06:44:46.221�[0m
�[1mSTEP:�[0m Found 62 events. �[38;5;243m12/29/22 06:44:46.255�[0m
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:00 +0000 UTC - event for failed-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job failed-jobs-history-limit-27871600
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:00 +0000 UTC - event for failed-jobs-history-limit-27871600: {job-controller } SuccessfulCreate: Created pod: failed-jobs-history-limit-27871600-p8jnr
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:00 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {default-scheduler } Scheduled: Successfully assigned cronjob-8627/failed-jobs-history-limit-27871600-p8jnr to i-07d5c844ec1aba791
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:00 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "702607d729881f66faffc0c7c3a5f45070b416a087430b1dd26c536308d39b3e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:11 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "28fdb81b957146962a1f5b73f493f6b0ab6a5c2c22e714d3ed2520ba33dd00c7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:22 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f7b9d5ae2a986aed93b0d703d5c8e4c716c6bfbfd9ef9dbafd242ccc687c8f7e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:35 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1aa3dd4a0cd0e152a8412524504cd90b679c1db19213958792737c805e017796": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:40:49 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3d6137cb5297472536c7e1109e732b634e4e0d15a67b11b76a73e4316382b82a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:00 +0000 UTC - event for failed-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job failed-jobs-history-limit-27871601
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:00 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "56fb5e597442f4d12b9f6c6b075ed1e3faa4f1eeb8575c8e0324df8695a3ff2f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:00 +0000 UTC - event for failed-jobs-history-limit-27871601: {job-controller } SuccessfulCreate: Created pod: failed-jobs-history-limit-27871601-lg7kl
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:00 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b6729b72e98b66b78fda8916660fed259181d51a01343a84e6393d24964a4388": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:00 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {default-scheduler } Scheduled: Successfully assigned cronjob-8627/failed-jobs-history-limit-27871601-lg7kl to i-07d5c844ec1aba791
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:13 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "343be5e360f8980ee981c81099744787dfc23f48a93ae749b5b18d9e5404cf8e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:14 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d7560019f44658067e9bd370652e65fd25a2e770d039dc7d3f3f4ac9a8a63f99": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:26 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "371030c3de9e742b60a6ff5097b7937beffb359d68ef1275c5b9aa0778016e50": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:29 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d5e3380da5f2af15916cab5262ac30ed2d5a7a45d9f3ec0f64baa4be0beca096": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:41 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "748089956c05caecd31f9f69b7a52bcc03815efdf5f3ff93abd518ff9e936f7b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:41 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3155b5d0a1e0dd257658906b951bb70137e634624d77889aed97507a67e1caed": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:53 +0000 UTC - event for failed-jobs-history-limit-27871600-p8jnr: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4297b8088ab14ffebca258a0e886f2bee60f64c664d14c9e2d00d86f59cfef5d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:41:57 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d2e9e560b3ea36be09a310da8d63d3cfcc817ba5ec8a7776f856ded30a8d893f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:00 +0000 UTC - event for failed-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job failed-jobs-history-limit-27871602
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:00 +0000 UTC - event for failed-jobs-history-limit-27871602: {job-controller } SuccessfulCreate: Created pod: failed-jobs-history-limit-27871602-87bxk
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:00 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {default-scheduler } Scheduled: Successfully assigned cronjob-8627/failed-jobs-history-limit-27871602-87bxk to i-07d5c844ec1aba791
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:00 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dab53956dab28d630a33bb9d72e62bb06983e6bd044809088e0691b3d8005f2c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:12 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3133d0c1fbb65be70d73c89e1f78a8bd06d908a57a1ddaca5b538897ea111a6e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:15 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8d44dc7288bbcbe854fd87c4366868372d792666f7e239adc6c4708814f36f57": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:25 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "53c9b6f5c318a049af12a80163a1808de29e5dee098897b9f499efa2920dba00": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:28 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e75bfccd985b80ef26f75082690de5f54b40e13edc5b2d2b241bf881a5d32018": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:39 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bfb417c44d75cf54d04c7fd10e0bd08be84551903502ae17b14fab06e3b4122c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:39 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7d40b9d41a9ba68f5d9e8f25c884739cb0d7170461881ea54c59abcc7d36b9d9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:51 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cfcb7addb0100ea2b83a92a6618f90b0f9136df666114ea4cbb912c35a430b8e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:42:54 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bb1abc805c5f87f7a8c02c4b37aef6fc1c3318eee0577e8f4e0e57b0de43fc31": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:00 +0000 UTC - event for failed-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job failed-jobs-history-limit-27871603
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:00 +0000 UTC - event for failed-jobs-history-limit-27871603: {job-controller } SuccessfulCreate: Created pod: failed-jobs-history-limit-27871603-sr5mg
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:00 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {default-scheduler } Scheduled: Successfully assigned cronjob-8627/failed-jobs-history-limit-27871603-sr5mg to i-07d5c844ec1aba791
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:00 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "eb3d4c80c80cf4688281f2b79f366a2ff1a1f173722844bbab0193741bb1c64e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:06 +0000 UTC - event for failed-jobs-history-limit-27871601-lg7kl: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "40112c6250f0e01fef49eb97b5cda301a1d9306654a686982d7eb9b5740ea521": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:08 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1fa05e61085dfa5a9a2f662a3b29422ad81f65fcb17e295df414b057dd2a02a3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:16 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2619ef23e73443f9cac23dd6104eda846e8d49573970d9ad1aeb3d7ff557db6c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:21 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c48b7cf026a934a141872cd057cc140ac8fcebe1e0558610173523c7d929d194": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:30 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "77827dabc940ebfe128ba8f952194bd47ea458aeb4a8c0fc330b72fbabc27ac4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:33 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7584c30be6075d458b5b88a618aee0aab53fdd026d960b5ea5010dcfa0ea72c0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:41 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "01206d5fdc4c2e9e23cf5e7a057a0a84330b46811d84a649284a8c6cbadc652e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:44 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fa0cd0ee807a6906d4361fd73581a61fa9d35c12ca1727f3fa008816cd53e19c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:53 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b04b17ea117c8d3383c6e72854288f52a0e37ca93e3daa1d7f684aee78c681c6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:43:55 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "85b1cecd32560da8bf8d15082d4173b5c024ba08d9627f99aaa10dee3f921936": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:00 +0000 UTC - event for failed-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job failed-jobs-history-limit-27871604
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:00 +0000 UTC - event for failed-jobs-history-limit-27871604: {job-controller } SuccessfulCreate: Created pod: failed-jobs-history-limit-27871604-jqgj9
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:00 +0000 UTC - event for failed-jobs-history-limit-27871604-jqgj9: {default-scheduler } Scheduled: Successfully assigned cronjob-8627/failed-jobs-history-limit-27871604-jqgj9 to i-07d5c844ec1aba791
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:00 +0000 UTC - event for failed-jobs-history-limit-27871604-jqgj9: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bccc37ae6103ef29a79c327c78103f14178e8ad9ca4643a2ab2336ed62698787": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:05 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6c070a6df2d0cf827c5fc398023b894a98b9ad083be81cb108aeb84fb93e6487": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:12 +0000 UTC - event for failed-jobs-history-limit-27871604-jqgj9: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "862fe43fb8428e9fd59e418dcca1dafe5c4c23ea24beda452de3d2b3764073b5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:16 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "976564ca01cce9ddb38d212aac5c4cf5da9ce54e3d619ccebd83e15365e1e62d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:25 +0000 UTC - event for failed-jobs-history-limit-27871604-jqgj9: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f288af110a3457d03f5665606813ccfbc086868b0af9cd8df868336a7fda3ae7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:31 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "24c4852a9c64c5d957ff5f181e81720663c25b284fb854d3ab645cc62393c977": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:40 +0000 UTC - event for failed-jobs-history-limit-27871604-jqgj9: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a19bdcc8a673284f0ecd4825cb5369d0a593d3e5c2782c735c9939ecf3d8f65d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:45 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} Started: Started container c
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:45 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} Created: Created container c
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:45 +0000 UTC - event for failed-jobs-history-limit-27871603-sr5mg: {kubelet i-07d5c844ec1aba791} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:46 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} Created: Created container c
Dec 29 06:44:46.255: INFO: At 2022-12-29 06:44:46 +0000 UTC - event for failed-jobs-history-limit-27871602-87bxk: {kubelet i-07d5c844ec1aba791} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Dec 29 06:44:46.285: INFO: POD NODE PHASE GRACE CONDITIONS
Dec 29 06:44:46.285: INFO: failed-jobs-history-limit-27871600-p8jnr i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:40:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:40:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:40:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:40:00 +0000 UTC }]
Dec 29 06:44:46.285: INFO: failed-jobs-history-limit-27871601-lg7kl i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:41:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:41:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:41:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:41:00 +0000 UTC }]
Dec 29 06:44:46.285: INFO: failed-jobs-history-limit-27871602-87bxk i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:42:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:42:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:42:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:42:00 +0000 UTC }]
Dec 29 06:44:46.285: INFO: failed-jobs-history-limit-27871603-sr5mg i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:43:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:43:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:43:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:43:00 +0000 UTC }]
Dec 29 06:44:46.285: INFO: failed-jobs-history-limit-27871604-jqgj9 i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:44:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:44:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:44:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:44:00 +0000 UTC }]
Dec 29 06:44:46.285: INFO:
Dec 29 06:44:46.321: INFO: Unable to fetch cronjob-8627/failed-jobs-history-limit-27871600-p8jnr/c logs: the server rejected our request for an unknown reason (get pods failed-jobs-history-limit-27871600-p8jnr)
Dec 29 06:44:46.356: INFO: Unable to fetch cronjob-8627/failed-jobs-history-limit-27871601-lg7kl/c logs: the server rejected our request for an unknown reason (get pods failed-jobs-history-limit-27871601-lg7kl)
Dec 29 06:44:46.393: INFO: Unable to fetch cronjob-8627/failed-jobs-history-limit-27871602-87bxk/c logs: the server rejected our request for an unknown reason (get pods failed-jobs-history-limit-27871602-87bxk)
Dec 29 06:44:46.425: INFO: Unable to fetch cronjob-8627/failed-jobs-history-limit-27871603-sr5mg/c logs: the server rejected our request for an unknown reason (get pods failed-jobs-history-limit-27871603-sr5mg)
Dec 29 06:44:46.462: INFO: Unable to fetch cronjob-8627/failed-jobs-history-limit-27871604-jqgj9/c logs: the server rejected our request for an unknown reason (get pods failed-jobs-history-limit-27871604-jqgj9)
Dec 29 06:44:46.493: INFO:
Logging node info for node i-0345e6864820ace32
Dec 29 06:44:46.522: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 22567 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799,DevicePath:,},},Config:nil,},}
Dec 29 06:44:46.523: INFO:
Logging kubelet events for node i-0345e6864820ace32
Dec 29 06:44:46.554: INFO:
Logging pods the kubelet thinks is on node i-0345e6864820ace32
Dec 29 06:44:46.593: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:46.593: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:46.593: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:46.593: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container webserver ready: true, restart count 0
Dec 29 06:44:46.593: INFO: hostexec-i-0345e6864820ace32-29fpj started at 2022-12-29 06:42:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:44:46.593: INFO: simpletest.rc-gdhwc started at 2022-12-29 06:44:20 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container nginx ready: false, restart count 0
Dec 29 06:44:46.593: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:46.593: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:46.593: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:46.593: INFO: netserver-0 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:46.593: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:46.593: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container coredns ready: true, restart count 0
Dec 29 06:44:46.593: INFO: pod-subpath-test-preprovisionedpv-5b4m started at 2022-12-29 06:42:39 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Init container init-volume-preprovisionedpv-5b4m ready: false, restart count 0
Dec 29 06:44:46.593: INFO: Container test-container-subpath-preprovisionedpv-5b4m ready: false, restart count 0
Dec 29 06:44:46.593: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:46.593: INFO: netserver-0 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:46.593: INFO: netserver-0 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:46.593: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:46.593: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:46.593: INFO: inline-volume-tester2-lkk7j started at 2022-12-29 06:41:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.593: INFO: Container csi-volume-tester ready: false, restart count 0
Dec 29 06:44:46.748: INFO:
Latency metrics for node i-0345e6864820ace32
Dec 29 06:44:46.748: INFO:
Logging node info for node i-06be1dd8c97acf782
Dec 29 06:44:46.778: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 21540 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:41:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:44:46.779: INFO:
Logging kubelet events for node i-06be1dd8c97acf782
Dec 29 06:44:46.812: INFO:
Logging pods the kubelet thinks is on node i-06be1dd8c97acf782
Dec 29 06:44:46.848: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container etcd-manager ready: true, restart count 0
Dec 29 06:44:46.848: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container kube-scheduler ready: true, restart count 0
Dec 29 06:44:46.848: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container dns-controller ready: true, restart count 0
Dec 29 06:44:46.848: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container cilium-operator ready: true, restart count 0
Dec 29 06:44:46.848: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Dec 29 06:44:46.848: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container etcd-manager ready: true, restart count 0
Dec 29 06:44:46.848: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container healthcheck ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container kube-apiserver ready: true, restart count 1
Dec 29 06:44:46.848: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container kube-controller-manager ready: true, restart count 2
Dec 29 06:44:46.848: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:46.848: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:46.848: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container kops-controller ready: true, restart count 0
Dec 29 06:44:46.848: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded)
Dec 29 06:44:46.848: INFO: Container csi-attacher ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:46.848: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:47.023: INFO:
Latency metrics for node i-06be1dd8c97acf782
Dec 29 06:44:47.023: INFO:
Logging node info for node i-07228a57ea47d842b
Dec 29 06:44:47.053: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 22569 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2022-12-29 06:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9651^9428e159-8743-11ed-b070-a6882e2b2850],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:44:47.053: INFO:
Logging kubelet events for node i-07228a57ea47d842b
Dec 29 06:44:47.090: INFO:
Logging pods the kubelet thinks is on node i-07228a57ea47d842b
Dec 29 06:44:47.128: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container autoscaler ready: true, restart count 0
Dec 29 06:44:47.128: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:44:47.128: INFO: Container driver-registrar ready: true, restart count 0
Dec 29 06:44:47.128: INFO: Container mock ready: true, restart count 0
Dec 29 06:44:47.128: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.128: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.128: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.128: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:44:47.128: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:47.128: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:47.128: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:44:47.128: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.128: INFO: pod-2526fd38-277f-499e-b2c5-b55842c84c2d started at 2022-12-29 06:39:55 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container write-pod ready: false, restart count 0
Dec 29 06:44:47.128: INFO: netserver-1 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.128: INFO: netserver-1 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.128: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container coredns ready: true, restart count 0
Dec 29 06:44:47.128: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:44:47.128: INFO: netserver-1 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.128: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:47.128: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:47.128: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:47.128: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.128: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.128: INFO: Container volume-tester ready: false, restart count 0
Dec 29 06:44:47.309: INFO:
Latency metrics for node i-07228a57ea47d842b
Dec 29 06:44:47.309: INFO:
Logging node info for node i-07d5c844ec1aba791
Dec 29 06:44:47.339: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 22706 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:44:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:44:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},},Config:nil,},}
Dec 29 06:44:47.339: INFO:
Logging kubelet events for node i-07d5c844ec1aba791
Dec 29 06:44:47.376: INFO:
Logging pods the kubelet thinks is on node i-07d5c844ec1aba791
Dec 29 06:44:47.426: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.427: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:47.427: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:47.427: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:47.427: INFO: simpletest.rc-lg82l started at 2022-12-29 06:44:20 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container nginx ready: false, restart count 0
Dec 29 06:44:47.427: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container webserver ready: true, restart count 0
Dec 29 06:44:47.427: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:47.427: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:47.427: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:44:47.427: INFO: pod-projected-configmaps-03cdd563-f67f-480c-8bad-23ae8f2d256a started at 2022-12-29 06:42:28 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container createcm-volume-test ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container delcm-volume-test ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container updcm-volume-test ready: false, restart count 0
Dec 29 06:44:47.427: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.427: INFO: busybox-3b39c30d-31e2-49ad-81c2-e1e37a94c517 started at 2022-12-29 06:39:50 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container busybox ready: false, restart count 0
Dec 29 06:44:47.427: INFO: e2e-k9hf4-48rpj started at <nil> (0+0 container statuses recorded)
Dec 29 06:44:47.427: INFO: failed-jobs-history-limit-27871602-87bxk started at 2022-12-29 06:42:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container c ready: false, restart count 0
Dec 29 06:44:47.427: INFO: pfpod started at 2022-12-29 06:44:38 +0000 UTC (0+2 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container portforwardtester ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container readiness ready: false, restart count 0
Dec 29 06:44:47.427: INFO: hostpath-symlink-prep-provisioning-3263 started at 2022-12-29 06:39:49 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container init-volume-provisioning-3263 ready: false, restart count 0
Dec 29 06:44:47.427: INFO: failed-jobs-history-limit-27871601-lg7kl started at 2022-12-29 06:41:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container c ready: false, restart count 0
Dec 29 06:44:47.427: INFO: failed-jobs-history-limit-27871603-sr5mg started at 2022-12-29 06:43:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container c ready: false, restart count 0
Dec 29 06:44:47.427: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.427: INFO: failed-jobs-history-limit-27871600-p8jnr started at 2022-12-29 06:40:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container c ready: false, restart count 0
Dec 29 06:44:47.427: INFO: hostpath-injector started at 2022-12-29 06:40:02 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container hostpath-injector ready: false, restart count 0
Dec 29 06:44:47.427: INFO: netserver-2 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.427: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.427: INFO: failed-jobs-history-limit-27871604-jqgj9 started at 2022-12-29 06:44:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container c ready: false, restart count 0
Dec 29 06:44:47.427: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.427: INFO: netserver-2 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.427: INFO: sample-webhook-deployment-865554f4d9-n7km6 started at 2022-12-29 06:41:01 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container sample-webhook ready: false, restart count 0
Dec 29 06:44:47.427: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.427: INFO: httpd started at 2022-12-29 06:42:01 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container httpd ready: false, restart count 0
Dec 29 06:44:47.427: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:44:32 +0000 UTC (0+7 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container csi-resizer ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container csi-snapshotter ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container hostpath ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container liveness-probe ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container node-driver-registrar ready: false, restart count 0
Dec 29 06:44:47.427: INFO: e2e-k9hf4-5ldkk started at <nil> (0+0 container statuses recorded)
Dec 29 06:44:47.427: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container jessie-querier ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container querier ready: false, restart count 0
Dec 29 06:44:47.427: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.427: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container csi-volume-tester ready: true, restart count 0
Dec 29 06:44:47.427: INFO: netserver-2 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.427: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.702: INFO:
Latency metrics for node i-07d5c844ec1aba791
Dec 29 06:44:47.702: INFO:
Logging node info for node i-0b6ab0d2cddb81acd
Dec 29 06:44:47.733: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 22981 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7083":"i-0b6ab0d2cddb81acd","ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2022-12-29 06:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:44:47.733: INFO:
Logging kubelet events for node i-0b6ab0d2cddb81acd
Dec 29 06:44:47.767: INFO:
Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd
Dec 29 06:44:47.805: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:47.805: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:47.805: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:47.805: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:47.805: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:47.805: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:44:47.805: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.805: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:44:47.805: INFO: Container driver-registrar ready: false, restart count 0
Dec 29 06:44:47.805: INFO: Container mock ready: false, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.805: INFO: netserver-3 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.805: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:44:47.805: INFO: Container driver-registrar ready: false, restart count 0
Dec 29 06:44:47.805: INFO: Container mock ready: false, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.805: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container donothing ready: false, restart count 0
Dec 29 06:44:47.805: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:47.805: INFO: netserver-3 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.805: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:44:47.805: INFO: netserver-3 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:47.805: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:47.979: INFO:
Latency metrics for node i-0b6ab0d2cddb81acd
[DeferCleanup (Each)] [sig-apps] CronJob
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "cronjob-8627" for this suite. �[38;5;243m12/29/22 06:44:47.979�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sDisruptionController\sevictions\:\senough\spods\,\sabsolute\s\=\>\sshould\sallow\san\seviction$'
test/e2e/apps/disruption.go:580
k8s.io/kubernetes/test/e2e/apps.waitForPodsOrDie({0x801e128?, 0xc003721380}, {0xc0000854c0, 0xf}, 0x3)
test/e2e/apps/disruption.go:580 +0x167
k8s.io/kubernetes/test/e2e/apps.glob..func6.7()
test/e2e/apps/disruption.go:326 +0x49f
[BeforeEach] [sig-apps] DisruptionController
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:34:23.795�[0m
Dec 29 06:34:23.796: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename disruption �[38;5;243m12/29/22 06:34:23.797�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:34:23.887�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:34:23.946�[0m
[BeforeEach] [sig-apps] DisruptionController
test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-apps] DisruptionController
test/e2e/apps/disruption.go:72
[It] evictions: enough pods, absolute => should allow an eviction
test/e2e/apps/disruption.go:290
�[1mSTEP:�[0m Waiting for the pdb to be processed �[38;5;243m12/29/22 06:34:24.14�[0m
�[1mSTEP:�[0m locating a running pod �[38;5;243m12/29/22 06:34:24.17�[0m
�[1mSTEP:�[0m Waiting for all pods to be running �[38;5;243m12/29/22 06:34:36.231�[0m
Dec 29 06:34:36.261: INFO: running pods: 1 < 3
Dec 29 06:34:38.292: INFO: running pods: 1 < 3
Dec 29 06:34:40.291: INFO: running pods: 1 < 3
Dec 29 06:34:42.292: INFO: running pods: 2 < 3
Dec 29 06:34:44.292: INFO: running pods: 2 < 3
Dec 29 06:34:46.292: INFO: running pods: 2 < 3
Dec 29 06:34:48.292: INFO: running pods: 2 < 3
Dec 29 06:34:50.292: INFO: running pods: 2 < 3
Dec 29 06:34:52.292: INFO: running pods: 2 < 3
Dec 29 06:34:54.292: INFO: running pods: 2 < 3
Dec 29 06:34:56.292: INFO: running pods: 2 < 3
Dec 29 06:34:58.293: INFO: running pods: 2 < 3
Dec 29 06:35:00.292: INFO: running pods: 2 < 3
Dec 29 06:35:02.292: INFO: running pods: 2 < 3
Dec 29 06:35:04.292: INFO: running pods: 2 < 3
Dec 29 06:35:06.292: INFO: running pods: 2 < 3
Dec 29 06:35:08.292: INFO: running pods: 2 < 3
Dec 29 06:35:10.292: INFO: running pods: 2 < 3
Dec 29 06:35:12.292: INFO: running pods: 2 < 3
Dec 29 06:35:14.291: INFO: running pods: 2 < 3
Dec 29 06:35:16.292: INFO: running pods: 2 < 3
Dec 29 06:35:18.293: INFO: running pods: 2 < 3
Dec 29 06:35:20.292: INFO: running pods: 2 < 3
Dec 29 06:35:22.292: INFO: running pods: 2 < 3
Dec 29 06:35:24.292: INFO: running pods: 2 < 3
Dec 29 06:35:26.292: INFO: running pods: 2 < 3
Dec 29 06:35:28.292: INFO: running pods: 2 < 3
Dec 29 06:35:30.292: INFO: running pods: 2 < 3
Dec 29 06:35:32.293: INFO: running pods: 2 < 3
Dec 29 06:35:34.292: INFO: running pods: 2 < 3
Dec 29 06:35:36.292: INFO: running pods: 2 < 3
Dec 29 06:35:38.295: INFO: running pods: 2 < 3
Dec 29 06:35:40.292: INFO: running pods: 2 < 3
Dec 29 06:35:42.292: INFO: running pods: 2 < 3
Dec 29 06:35:44.292: INFO: running pods: 2 < 3
Dec 29 06:35:46.291: INFO: running pods: 2 < 3
Dec 29 06:35:48.292: INFO: running pods: 2 < 3
Dec 29 06:35:50.292: INFO: running pods: 2 < 3
Dec 29 06:35:52.292: INFO: running pods: 2 < 3
Dec 29 06:35:54.292: INFO: running pods: 2 < 3
Dec 29 06:35:56.292: INFO: running pods: 2 < 3
Dec 29 06:35:58.292: INFO: running pods: 2 < 3
Dec 29 06:36:00.292: INFO: running pods: 2 < 3
Dec 29 06:36:02.292: INFO: running pods: 2 < 3
Dec 29 06:36:04.292: INFO: running pods: 2 < 3
Dec 29 06:36:06.292: INFO: running pods: 2 < 3
Dec 29 06:36:08.292: INFO: running pods: 2 < 3
Dec 29 06:36:10.291: INFO: running pods: 2 < 3
Dec 29 06:36:12.293: INFO: running pods: 2 < 3
Dec 29 06:36:14.292: INFO: running pods: 2 < 3
Dec 29 06:36:16.292: INFO: running pods: 2 < 3
Dec 29 06:36:18.291: INFO: running pods: 2 < 3
Dec 29 06:36:20.293: INFO: running pods: 2 < 3
Dec 29 06:36:22.292: INFO: running pods: 2 < 3
Dec 29 06:36:24.292: INFO: running pods: 2 < 3
Dec 29 06:36:26.292: INFO: running pods: 2 < 3
Dec 29 06:36:28.293: INFO: running pods: 2 < 3
Dec 29 06:36:30.292: INFO: running pods: 2 < 3
Dec 29 06:36:32.293: INFO: running pods: 2 < 3
Dec 29 06:36:34.292: INFO: running pods: 2 < 3
Dec 29 06:36:36.291: INFO: running pods: 2 < 3
Dec 29 06:36:38.292: INFO: running pods: 2 < 3
Dec 29 06:36:40.292: INFO: running pods: 2 < 3
Dec 29 06:36:42.292: INFO: running pods: 2 < 3
Dec 29 06:36:44.293: INFO: running pods: 2 < 3
Dec 29 06:36:46.292: INFO: running pods: 2 < 3
Dec 29 06:36:48.292: INFO: running pods: 2 < 3
Dec 29 06:36:50.292: INFO: running pods: 2 < 3
Dec 29 06:36:52.291: INFO: running pods: 2 < 3
Dec 29 06:36:54.292: INFO: running pods: 2 < 3
Dec 29 06:36:56.292: INFO: running pods: 2 < 3
Dec 29 06:36:58.293: INFO: running pods: 2 < 3
Dec 29 06:37:00.293: INFO: running pods: 2 < 3
Dec 29 06:37:02.292: INFO: running pods: 2 < 3
Dec 29 06:37:04.292: INFO: running pods: 2 < 3
Dec 29 06:37:06.292: INFO: running pods: 2 < 3
Dec 29 06:37:08.292: INFO: running pods: 2 < 3
Dec 29 06:37:10.293: INFO: running pods: 2 < 3
Dec 29 06:37:12.292: INFO: running pods: 2 < 3
Dec 29 06:37:14.292: INFO: running pods: 2 < 3
Dec 29 06:37:16.292: INFO: running pods: 2 < 3
Dec 29 06:37:18.292: INFO: running pods: 2 < 3
Dec 29 06:37:20.292: INFO: running pods: 2 < 3
Dec 29 06:37:22.294: INFO: running pods: 2 < 3
Dec 29 06:37:24.292: INFO: running pods: 2 < 3
Dec 29 06:37:26.292: INFO: running pods: 2 < 3
Dec 29 06:37:28.291: INFO: running pods: 2 < 3
Dec 29 06:37:30.292: INFO: running pods: 2 < 3
Dec 29 06:37:32.293: INFO: running pods: 2 < 3
Dec 29 06:37:34.292: INFO: running pods: 2 < 3
Dec 29 06:37:36.293: INFO: running pods: 2 < 3
Dec 29 06:37:38.291: INFO: running pods: 2 < 3
Dec 29 06:37:40.292: INFO: running pods: 2 < 3
Dec 29 06:37:42.292: INFO: running pods: 2 < 3
Dec 29 06:37:44.292: INFO: running pods: 2 < 3
Dec 29 06:37:46.291: INFO: running pods: 2 < 3
Dec 29 06:37:48.292: INFO: running pods: 2 < 3
Dec 29 06:37:50.292: INFO: running pods: 2 < 3
Dec 29 06:37:52.292: INFO: running pods: 2 < 3
Dec 29 06:37:54.292: INFO: running pods: 2 < 3
Dec 29 06:37:56.291: INFO: running pods: 2 < 3
Dec 29 06:37:58.292: INFO: running pods: 2 < 3
Dec 29 06:38:00.293: INFO: running pods: 2 < 3
Dec 29 06:38:02.292: INFO: running pods: 2 < 3
Dec 29 06:38:04.292: INFO: running pods: 2 < 3
Dec 29 06:38:06.292: INFO: running pods: 2 < 3
Dec 29 06:38:08.292: INFO: running pods: 2 < 3
Dec 29 06:38:10.292: INFO: running pods: 2 < 3
Dec 29 06:38:12.292: INFO: running pods: 2 < 3
Dec 29 06:38:14.292: INFO: running pods: 2 < 3
Dec 29 06:38:16.294: INFO: running pods: 2 < 3
Dec 29 06:38:18.291: INFO: running pods: 2 < 3
Dec 29 06:38:20.292: INFO: running pods: 2 < 3
Dec 29 06:38:22.292: INFO: running pods: 2 < 3
Dec 29 06:38:24.292: INFO: running pods: 2 < 3
Dec 29 06:38:26.292: INFO: running pods: 2 < 3
Dec 29 06:38:28.294: INFO: running pods: 2 < 3
Dec 29 06:38:30.293: INFO: running pods: 2 < 3
Dec 29 06:38:32.292: INFO: running pods: 2 < 3
Dec 29 06:38:34.292: INFO: running pods: 2 < 3
Dec 29 06:38:36.292: INFO: running pods: 2 < 3
Dec 29 06:38:38.292: INFO: running pods: 2 < 3
Dec 29 06:38:40.293: INFO: running pods: 2 < 3
Dec 29 06:38:42.292: INFO: running pods: 2 < 3
Dec 29 06:38:44.292: INFO: running pods: 2 < 3
Dec 29 06:38:46.293: INFO: running pods: 2 < 3
Dec 29 06:38:48.293: INFO: running pods: 2 < 3
Dec 29 06:38:50.292: INFO: running pods: 2 < 3
Dec 29 06:38:52.292: INFO: running pods: 2 < 3
Dec 29 06:38:54.292: INFO: running pods: 2 < 3
Dec 29 06:38:56.292: INFO: running pods: 2 < 3
Dec 29 06:38:58.292: INFO: running pods: 2 < 3
Dec 29 06:39:00.294: INFO: running pods: 2 < 3
Dec 29 06:39:02.292: INFO: running pods: 2 < 3
Dec 29 06:39:04.292: INFO: running pods: 2 < 3
Dec 29 06:39:06.292: INFO: running pods: 2 < 3
Dec 29 06:39:08.293: INFO: running pods: 2 < 3
Dec 29 06:39:10.291: INFO: running pods: 2 < 3
Dec 29 06:39:12.295: INFO: running pods: 2 < 3
Dec 29 06:39:14.292: INFO: running pods: 2 < 3
Dec 29 06:39:16.292: INFO: running pods: 2 < 3
Dec 29 06:39:18.293: INFO: running pods: 2 < 3
Dec 29 06:39:20.292: INFO: running pods: 2 < 3
Dec 29 06:39:22.292: INFO: running pods: 2 < 3
Dec 29 06:39:24.292: INFO: running pods: 2 < 3
Dec 29 06:39:26.292: INFO: running pods: 2 < 3
Dec 29 06:39:28.292: INFO: running pods: 2 < 3
Dec 29 06:39:30.294: INFO: running pods: 2 < 3
Dec 29 06:39:32.293: INFO: running pods: 2 < 3
Dec 29 06:39:34.295: INFO: running pods: 2 < 3
Dec 29 06:39:36.292: INFO: running pods: 2 < 3
Dec 29 06:39:38.292: INFO: running pods: 2 < 3
Dec 29 06:39:40.291: INFO: running pods: 2 < 3
Dec 29 06:39:42.292: INFO: running pods: 2 < 3
Dec 29 06:39:44.294: INFO: running pods: 2 < 3
Dec 29 06:39:46.291: INFO: running pods: 2 < 3
Dec 29 06:39:48.292: INFO: running pods: 2 < 3
Dec 29 06:39:50.292: INFO: running pods: 2 < 3
Dec 29 06:39:52.293: INFO: running pods: 2 < 3
Dec 29 06:39:54.292: INFO: running pods: 2 < 3
Dec 29 06:39:56.292: INFO: running pods: 2 < 3
Dec 29 06:39:58.292: INFO: running pods: 2 < 3
Dec 29 06:40:00.293: INFO: running pods: 2 < 3
Dec 29 06:40:02.292: INFO: running pods: 2 < 3
Dec 29 06:40:04.292: INFO: running pods: 2 < 3
Dec 29 06:40:06.292: INFO: running pods: 2 < 3
Dec 29 06:40:08.292: INFO: running pods: 2 < 3
Dec 29 06:40:10.292: INFO: running pods: 2 < 3
Dec 29 06:40:12.293: INFO: running pods: 2 < 3
Dec 29 06:40:14.292: INFO: running pods: 2 < 3
Dec 29 06:40:16.292: INFO: running pods: 2 < 3
Dec 29 06:40:18.293: INFO: running pods: 2 < 3
Dec 29 06:40:20.292: INFO: running pods: 2 < 3
Dec 29 06:40:22.292: INFO: running pods: 2 < 3
Dec 29 06:40:24.292: INFO: running pods: 2 < 3
Dec 29 06:40:26.293: INFO: running pods: 2 < 3
Dec 29 06:40:28.291: INFO: running pods: 2 < 3
Dec 29 06:40:30.292: INFO: running pods: 2 < 3
Dec 29 06:40:32.292: INFO: running pods: 2 < 3
Dec 29 06:40:34.291: INFO: running pods: 2 < 3
Dec 29 06:40:36.292: INFO: running pods: 2 < 3
Dec 29 06:40:38.293: INFO: running pods: 2 < 3
Dec 29 06:40:40.292: INFO: running pods: 2 < 3
Dec 29 06:40:42.292: INFO: running pods: 2 < 3
Dec 29 06:40:44.292: INFO: running pods: 2 < 3
Dec 29 06:40:46.292: INFO: running pods: 2 < 3
Dec 29 06:40:48.292: INFO: running pods: 2 < 3
Dec 29 06:40:50.292: INFO: running pods: 2 < 3
Dec 29 06:40:52.292: INFO: running pods: 2 < 3
Dec 29 06:40:54.292: INFO: running pods: 2 < 3
Dec 29 06:40:56.292: INFO: running pods: 2 < 3
Dec 29 06:40:58.293: INFO: running pods: 2 < 3
Dec 29 06:41:00.293: INFO: running pods: 2 < 3
Dec 29 06:41:02.292: INFO: running pods: 2 < 3
Dec 29 06:41:04.292: INFO: running pods: 2 < 3
Dec 29 06:41:06.292: INFO: running pods: 2 < 3
Dec 29 06:41:08.292: INFO: running pods: 2 < 3
Dec 29 06:41:10.292: INFO: running pods: 2 < 3
Dec 29 06:41:12.292: INFO: running pods: 2 < 3
Dec 29 06:41:14.292: INFO: running pods: 2 < 3
Dec 29 06:41:16.293: INFO: running pods: 2 < 3
Dec 29 06:41:18.291: INFO: running pods: 2 < 3
Dec 29 06:41:20.292: INFO: running pods: 2 < 3
Dec 29 06:41:22.292: INFO: running pods: 2 < 3
Dec 29 06:41:24.292: INFO: running pods: 2 < 3
Dec 29 06:41:26.293: INFO: running pods: 2 < 3
Dec 29 06:41:28.293: INFO: running pods: 2 < 3
Dec 29 06:41:30.292: INFO: running pods: 2 < 3
Dec 29 06:41:32.292: INFO: running pods: 2 < 3
Dec 29 06:41:34.292: INFO: running pods: 2 < 3
Dec 29 06:41:36.293: INFO: running pods: 2 < 3
Dec 29 06:41:38.292: INFO: running pods: 2 < 3
Dec 29 06:41:40.291: INFO: running pods: 2 < 3
Dec 29 06:41:42.291: INFO: running pods: 2 < 3
Dec 29 06:41:44.292: INFO: running pods: 2 < 3
Dec 29 06:41:46.291: INFO: running pods: 2 < 3
Dec 29 06:41:48.293: INFO: running pods: 2 < 3
Dec 29 06:41:50.292: INFO: running pods: 2 < 3
Dec 29 06:41:52.292: INFO: running pods: 2 < 3
Dec 29 06:41:54.292: INFO: running pods: 2 < 3
Dec 29 06:41:56.292: INFO: running pods: 2 < 3
Dec 29 06:41:58.292: INFO: running pods: 2 < 3
Dec 29 06:42:00.293: INFO: running pods: 2 < 3
Dec 29 06:42:02.292: INFO: running pods: 2 < 3
Dec 29 06:42:04.292: INFO: running pods: 2 < 3
Dec 29 06:42:06.294: INFO: running pods: 2 < 3
Dec 29 06:42:08.292: INFO: running pods: 2 < 3
Dec 29 06:42:10.292: INFO: running pods: 2 < 3
Dec 29 06:42:12.292: INFO: running pods: 2 < 3
Dec 29 06:42:14.292: INFO: running pods: 2 < 3
Dec 29 06:42:16.293: INFO: running pods: 2 < 3
Dec 29 06:42:18.294: INFO: running pods: 2 < 3
Dec 29 06:42:20.293: INFO: running pods: 2 < 3
Dec 29 06:42:22.293: INFO: running pods: 2 < 3
Dec 29 06:42:24.291: INFO: running pods: 2 < 3
Dec 29 06:42:26.292: INFO: running pods: 2 < 3
Dec 29 06:42:28.291: INFO: running pods: 2 < 3
Dec 29 06:42:30.292: INFO: running pods: 2 < 3
Dec 29 06:42:32.292: INFO: running pods: 2 < 3
Dec 29 06:42:34.292: INFO: running pods: 2 < 3
Dec 29 06:42:36.291: INFO: running pods: 2 < 3
Dec 29 06:42:38.292: INFO: running pods: 2 < 3
Dec 29 06:42:40.291: INFO: running pods: 2 < 3
Dec 29 06:42:42.292: INFO: running pods: 2 < 3
Dec 29 06:42:44.292: INFO: running pods: 2 < 3
Dec 29 06:42:46.292: INFO: running pods: 2 < 3
Dec 29 06:42:48.293: INFO: running pods: 2 < 3
Dec 29 06:42:50.292: INFO: running pods: 2 < 3
Dec 29 06:42:52.291: INFO: running pods: 2 < 3
Dec 29 06:42:54.292: INFO: running pods: 2 < 3
Dec 29 06:42:56.292: INFO: running pods: 2 < 3
Dec 29 06:42:58.293: INFO: running pods: 2 < 3
Dec 29 06:43:00.303: INFO: running pods: 2 < 3
Dec 29 06:43:02.292: INFO: running pods: 2 < 3
Dec 29 06:43:04.292: INFO: running pods: 2 < 3
Dec 29 06:43:06.291: INFO: running pods: 2 < 3
Dec 29 06:43:08.293: INFO: running pods: 2 < 3
Dec 29 06:43:10.291: INFO: running pods: 2 < 3
Dec 29 06:43:12.293: INFO: running pods: 2 < 3
Dec 29 06:43:14.292: INFO: running pods: 2 < 3
Dec 29 06:43:16.292: INFO: running pods: 2 < 3
Dec 29 06:43:18.292: INFO: running pods: 2 < 3
Dec 29 06:43:20.292: INFO: running pods: 2 < 3
Dec 29 06:43:22.292: INFO: running pods: 2 < 3
Dec 29 06:43:24.292: INFO: running pods: 2 < 3
Dec 29 06:43:26.292: INFO: running pods: 2 < 3
Dec 29 06:43:28.292: INFO: running pods: 2 < 3
Dec 29 06:43:30.293: INFO: running pods: 2 < 3
Dec 29 06:43:32.291: INFO: running pods: 2 < 3
Dec 29 06:43:34.292: INFO: running pods: 2 < 3
Dec 29 06:43:36.291: INFO: running pods: 2 < 3
Dec 29 06:43:38.292: INFO: running pods: 2 < 3
Dec 29 06:43:40.291: INFO: running pods: 2 < 3
Dec 29 06:43:42.292: INFO: running pods: 2 < 3
Dec 29 06:43:44.292: INFO: running pods: 2 < 3
Dec 29 06:43:46.292: INFO: running pods: 2 < 3
Dec 29 06:43:48.292: INFO: running pods: 2 < 3
Dec 29 06:43:50.292: INFO: running pods: 2 < 3
Dec 29 06:43:52.292: INFO: running pods: 2 < 3
Dec 29 06:43:54.292: INFO: running pods: 2 < 3
Dec 29 06:43:56.291: INFO: running pods: 2 < 3
Dec 29 06:43:58.293: INFO: running pods: 2 < 3
Dec 29 06:44:00.292: INFO: running pods: 2 < 3
Dec 29 06:44:02.291: INFO: running pods: 2 < 3
Dec 29 06:44:04.292: INFO: running pods: 2 < 3
Dec 29 06:44:06.292: INFO: running pods: 2 < 3
Dec 29 06:44:08.291: INFO: running pods: 2 < 3
Dec 29 06:44:10.291: INFO: running pods: 2 < 3
Dec 29 06:44:12.292: INFO: running pods: 2 < 3
Dec 29 06:44:14.292: INFO: running pods: 2 < 3
Dec 29 06:44:16.292: INFO: running pods: 2 < 3
Dec 29 06:44:18.291: INFO: running pods: 2 < 3
Dec 29 06:44:20.291: INFO: running pods: 2 < 3
Dec 29 06:44:22.291: INFO: running pods: 2 < 3
Dec 29 06:44:24.292: INFO: running pods: 2 < 3
Dec 29 06:44:26.291: INFO: running pods: 2 < 3
Dec 29 06:44:28.291: INFO: running pods: 2 < 3
Dec 29 06:44:30.292: INFO: running pods: 2 < 3
Dec 29 06:44:32.291: INFO: running pods: 2 < 3
Dec 29 06:44:34.291: INFO: running pods: 2 < 3
Dec 29 06:44:36.291: INFO: running pods: 2 < 3
Dec 29 06:44:36.321: INFO: running pods: 2 < 3
Dec 29 06:44:36.321: INFO: Unexpected error: Waiting for pods in namespace "disruption-7096" to be ready:
<*errors.errorString | 0xc000207d70>: {
s: "timed out waiting for the condition",
}
Dec 29 06:44:36.321: FAIL: Waiting for pods in namespace "disruption-7096" to be ready: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/apps.waitForPodsOrDie({0x801e128?, 0xc003721380}, {0xc0000854c0, 0xf}, 0x3)
test/e2e/apps/disruption.go:580 +0x167
k8s.io/kubernetes/test/e2e/apps.glob..func6.7()
test/e2e/apps/disruption.go:326 +0x49f
[AfterEach] [sig-apps] DisruptionController
test/e2e/framework/node/init/init.go:32
Dec 29 06:44:36.321: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] DisruptionController
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] DisruptionController
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:44:36.351�[0m
�[1mSTEP:�[0m Collecting events from namespace "disruption-7096". �[38;5;243m12/29/22 06:44:36.352�[0m
�[1mSTEP:�[0m Found 20 events. �[38;5;243m12/29/22 06:44:36.382�[0m
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:24 +0000 UTC - event for pod-0: {default-scheduler } Scheduled: Successfully assigned disruption-7096/pod-0 to i-07d5c844ec1aba791
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:24 +0000 UTC - event for pod-1: {default-scheduler } Scheduled: Successfully assigned disruption-7096/pod-1 to i-07d5c844ec1aba791
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:24 +0000 UTC - event for pod-2: {default-scheduler } Scheduled: Successfully assigned disruption-7096/pod-2 to i-0b6ab0d2cddb81acd
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:25 +0000 UTC - event for pod-0: {kubelet i-07d5c844ec1aba791} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:25 +0000 UTC - event for pod-0: {kubelet i-07d5c844ec1aba791} Created: Created container donothing
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:25 +0000 UTC - event for pod-0: {kubelet i-07d5c844ec1aba791} Started: Started container donothing
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:25 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "21354beab5f2ae6a5acbff0f3de7f0438e7f74db8a6d99f73808ae13668cbf3a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:25 +0000 UTC - event for pod-2: {kubelet i-0b6ab0d2cddb81acd} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7f85e99430f98db21c94faa2429c40ddf0bdb1d18fb209946925c2661d731070": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:38 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d61da0177f6779ca3652080a8c2f3de17b71d87899330121e1f3da7b289a6f99": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:40 +0000 UTC - event for pod-2: {kubelet i-0b6ab0d2cddb81acd} Started: Started container donothing
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:40 +0000 UTC - event for pod-2: {kubelet i-0b6ab0d2cddb81acd} Created: Created container donothing
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:40 +0000 UTC - event for pod-2: {kubelet i-0b6ab0d2cddb81acd} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:34:51 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bbe02cf61ece6078022373d8ab297f5319ff2ac09641ab98cad681a1f9ee4265": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:35:05 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "68bf0f1996b6169c6ca70ebe5d4e22acb1f62577b041021d7e6537c724c41818": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:35:16 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "752232ffe481f11f43e61e464df6d7a875ab7c667d44ab0b53df90536d6f04e3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:35:28 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "19ff061dd4e9efadd73da56701110c9826ecacb05b40131d0c8a38b6557c6d61": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:35:41 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bfb44ea07d374d0faf7cf56f952dd7d745c04d9317338f7469a7a9ddbf04975f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:35:54 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9753ed5b6500848ef6bf2f3153bd9a05b220035ab47797a05a737a56b6d45788": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:36:07 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0f50116ca65263538cfa9b938e444b350ecc6123f593b359b23d26c1cdc8b549": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.382: INFO: At 2022-12-29 06:36:21 +0000 UTC - event for pod-1: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5345c3a48cbdb30316a53aac8ffb14ec883558e8a912873c28c4d4248000c7c9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Dec 29 06:44:36.411: INFO: POD NODE PHASE GRACE CONDITIONS
Dec 29 06:44:36.411: INFO: pod-0 i-07d5c844ec1aba791 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:26 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:26 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC }]
Dec 29 06:44:36.411: INFO: pod-1 i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC ContainersNotReady containers with unready status: [donothing]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC ContainersNotReady containers with unready status: [donothing]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC }]
Dec 29 06:44:36.411: INFO: pod-2 i-0b6ab0d2cddb81acd Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:40 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:40 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:24 +0000 UTC }]
Dec 29 06:44:36.411: INFO:
Dec 29 06:44:36.473: INFO: Unable to fetch disruption-7096/pod-1/donothing logs: the server rejected our request for an unknown reason (get pods pod-1)
Dec 29 06:44:36.535: INFO:
Logging node info for node i-0345e6864820ace32
Dec 29 06:44:36.564: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 22567 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:42:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:42:01 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d5158fbd50453799,DevicePath:,},},Config:nil,},}
Dec 29 06:44:36.565: INFO:
Logging kubelet events for node i-0345e6864820ace32
Dec 29 06:44:36.596: INFO:
Logging pods the kubelet thinks is on node i-0345e6864820ace32
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: inline-volume-tester2-lkk7j started at 2022-12-29 06:41:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container csi-volume-tester ready: false, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-ck56n started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-mcnp2 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:36.635: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:36.635: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container webserver ready: true, restart count 0
Dec 29 06:44:36.635: INFO: hostexec-i-0345e6864820ace32-29fpj started at 2022-12-29 06:42:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:44:36.635: INFO: simpletest.rc-gdhwc started at 2022-12-29 06:44:20 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container nginx ready: false, restart count 0
Dec 29 06:44:36.635: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:36.635: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:36.635: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dwftm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvtcq started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container coredns ready: true, restart count 0
Dec 29 06:44:36.635: INFO: pod-subpath-test-preprovisionedpv-5b4m started at 2022-12-29 06:42:39 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Init container init-volume-preprovisionedpv-5b4m ready: false, restart count 0
Dec 29 06:44:36.635: INFO: Container test-container-subpath-preprovisionedpv-5b4m ready: false, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.635: INFO: netserver-0 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:36.635: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gtk84 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.635: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:36.806: INFO:
Latency metrics for node i-0345e6864820ace32
Dec 29 06:44:36.806: INFO:
Logging node info for node i-06be1dd8c97acf782
Dec 29 06:44:36.835: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 21540 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:41:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:41:19 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:44:36.835: INFO:
Logging kubelet events for node i-06be1dd8c97acf782
Dec 29 06:44:36.867: INFO:
Logging pods the kubelet thinks is on node i-06be1dd8c97acf782
Dec 29 06:44:36.902: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container kube-scheduler ready: true, restart count 0
Dec 29 06:44:36.902: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container dns-controller ready: true, restart count 0
Dec 29 06:44:36.902: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container cilium-operator ready: true, restart count 0
Dec 29 06:44:36.902: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Dec 29 06:44:36.902: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container etcd-manager ready: true, restart count 0
Dec 29 06:44:36.902: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container healthcheck ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container kube-apiserver ready: true, restart count 1
Dec 29 06:44:36.902: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container kube-controller-manager ready: true, restart count 2
Dec 29 06:44:36.902: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:36.902: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:36.902: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container kops-controller ready: true, restart count 0
Dec 29 06:44:36.902: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container csi-attacher ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:36.902: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:36.902: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:36.902: INFO: Container etcd-manager ready: true, restart count 0
Dec 29 06:44:37.069: INFO:
Latency metrics for node i-06be1dd8c97acf782
Dec 29 06:44:37.069: INFO:
Logging node info for node i-07228a57ea47d842b
Dec 29 06:44:37.099: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 22569 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2022-12-29 06:39:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:53 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9651^9428e159-8743-11ed-b070-a6882e2b2850],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:44:37.099: INFO:
Logging kubelet events for node i-07228a57ea47d842b
Dec 29 06:44:37.131: INFO:
Logging pods the kubelet thinks is on node i-07228a57ea47d842b
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: pod-2526fd38-277f-499e-b2c5-b55842c84c2d started at 2022-12-29 06:39:55 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container write-pod ready: false, restart count 0
Dec 29 06:44:37.168: INFO: netserver-1 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.168: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container coredns ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b9h6m started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4xmqc started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:37.168: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:37.168: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container volume-tester ready: false, restart count 0
Dec 29 06:44:37.168: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container autoscaler ready: true, restart count 0
Dec 29 06:44:37.168: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:44:37.168: INFO: Container driver-registrar ready: true, restart count 0
Dec 29 06:44:37.168: INFO: Container mock ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4l6sw started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-57b2m started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:37.168: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b6wc8 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.168: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.168: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:44:37.379: INFO:
Latency metrics for node i-07228a57ea47d842b
Dec 29 06:44:37.379: INFO:
Logging node info for node i-07d5c844ec1aba791
Dec 29 06:44:37.410: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 22571 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6856":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:44:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:44:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},},Config:nil,},}
Dec 29 06:44:37.410: INFO:
Logging kubelet events for node i-07d5c844ec1aba791
Dec 29 06:44:37.443: INFO:
Logging pods the kubelet thinks is on node i-07d5c844ec1aba791
Dec 29 06:44:37.482: INFO: hostpath-symlink-prep-provisioning-3263 started at 2022-12-29 06:39:49 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container init-volume-provisioning-3263 ready: false, restart count 0
Dec 29 06:44:37.482: INFO: failed-jobs-history-limit-27871601-lg7kl started at 2022-12-29 06:41:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container c ready: false, restart count 0
Dec 29 06:44:37.482: INFO: failed-jobs-history-limit-27871603-sr5mg started at 2022-12-29 06:43:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container c ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: failed-jobs-history-limit-27871600-p8jnr started at 2022-12-29 06:40:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container c ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-j7svz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:44:37.482: INFO: hostpath-injector started at 2022-12-29 06:40:02 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container hostpath-injector ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: failed-jobs-history-limit-27871604-jqgj9 started at 2022-12-29 06:44:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container c ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: pod-0 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container donothing ready: true, restart count 0
Dec 29 06:44:37.482: INFO: sample-webhook-deployment-865554f4d9-n7km6 started at 2022-12-29 06:41:01 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container sample-webhook ready: false, restart count 0
Dec 29 06:44:37.482: INFO: pod-1 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container donothing ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zlzqg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0
Dec 29 06:44:37.482: INFO: netserver-2 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.482: INFO: httpd started at 2022-12-29 06:42:01 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container httpd ready: false, restart count 0
Dec 29 06:44:37.482: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:44:32 +0000 UTC (0+7 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container csi-resizer ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container csi-snapshotter ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container hostpath ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container liveness-probe ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container node-driver-registrar ready: false, restart count 0
Dec 29 06:44:37.482: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container jessie-querier ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container querier ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.482: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container csi-volume-tester ready: true, restart count 0
Dec 29 06:44:37.482: INFO: netserver-2 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9v6vz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:37.482: INFO: simpletest.rc-lg82l started at 2022-12-29 06:44:20 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container nginx ready: false, restart count 0
Dec 29 06:44:37.482: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: pod-projected-configmaps-03cdd563-f67f-480c-8bad-23ae8f2d256a started at 2022-12-29 06:42:28 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container createcm-volume-test ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container delcm-volume-test ready: false, restart count 0
Dec 29 06:44:37.482: INFO: Container updcm-volume-test ready: false, restart count 0
Dec 29 06:44:37.482: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container webserver ready: true, restart count 0
Dec 29 06:44:37.482: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.482: INFO: busybox-3b39c30d-31e2-49ad-81c2-e1e37a94c517 started at 2022-12-29 06:39:50 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container busybox ready: false, restart count 0
Dec 29 06:44:37.482: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:33:49 +0000 UTC (0+7 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container csi-attacher ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container csi-provisioner ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container csi-snapshotter ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container hostpath ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:37.482: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:37.482: INFO: failed-jobs-history-limit-27871602-87bxk started at 2022-12-29 06:42:00 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.482: INFO: Container c ready: false, restart count 0
Dec 29 06:44:37.675: INFO:
Latency metrics for node i-07d5c844ec1aba791
Dec 29 06:44:37.675: INFO:
Logging node info for node i-0b6ab0d2cddb81acd
Dec 29 06:44:37.704: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 22573 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2022-12-29 06:44:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:44:31 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Dec 29 06:44:37.705: INFO:
Logging kubelet events for node i-0b6ab0d2cddb81acd
Dec 29 06:44:37.737: INFO:
Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container donothing ready: true, restart count 0
Dec 29 06:44:37.776: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:44:37.776: INFO: Container driver-registrar ready: false, restart count 0
Dec 29 06:44:37.776: INFO: Container mock ready: false, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: netserver-3 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.776: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container csi-attacher ready: false, restart count 0
Dec 29 06:44:37.776: INFO: netserver-3 started at 2022-12-29 06:44:37 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.776: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Init container clean-cilium-state ready: true, restart count 0
Dec 29 06:44:37.776: INFO: Container cilium-agent ready: true, restart count 0
Dec 29 06:44:37.776: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container ebs-plugin ready: true, restart count 0
Dec 29 06:44:37.776: INFO: Container liveness-probe ready: true, restart count 0
Dec 29 06:44:37.776: INFO: Container node-driver-registrar ready: true, restart count 0
Dec 29 06:44:37.776: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container csi-resizer ready: true, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d8lgx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-8qfzg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.776: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container csi-provisioner ready: false, restart count 0
Dec 29 06:44:37.776: INFO: Container driver-registrar ready: false, restart count 0
Dec 29 06:44:37.776: INFO: Container mock ready: false, restart count 0
Dec 29 06:44:37.776: INFO: local-client started at 2022-12-29 06:39:39 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container local-client ready: false, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: netserver-3 started at 2022-12-29 06:42:16 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container webserver ready: false, restart count 0
Dec 29 06:44:37.776: INFO: downwardapi-volume-cc6aea7e-266b-4611-a490-6cb3559eaf08 started at 2022-12-29 06:39:43 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container client-container ready: false, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6h6vh started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0
Dec 29 06:44:37.776: INFO: hostexec-i-0b6ab0d2cddb81acd-qppr8 started at 2022-12-29 06:34:30 +0000 UTC (0+1 container statuses recorded)
Dec 29 06:44:37.776: INFO: Container agnhost-container ready: true, restart count 0
Dec 29 06:44:37.957: INFO:
Latency metrics for node i-0b6ab0d2cddb81acd
[DeferCleanup (Each)] [sig-apps] DisruptionController
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "disruption-7096" for this suite. �[38;5;243m12/29/22 06:44:37.957�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\scontain\slast\sline\sof\sthe\slog$'
test/e2e/kubectl/kubectl.go:660 k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.9() test/e2e/kubectl/kubectl.go:660 +0x265from junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:33:54.991�[0m Dec 29 06:33:54.991: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m12/29/22 06:33:54.993�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:33:55.086�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:33:55.145�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [BeforeEach] Simple pod test/e2e/kubectl/kubectl.go:411 �[1mSTEP:�[0m creating the pod from �[38;5;243m12/29/22 06:33:55.204�[0m Dec 29 06:33:55.204: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2575 create -f -' Dec 29 06:33:56.551: INFO: stderr: "" Dec 29 06:33:56.551: INFO: stdout: "pod/httpd created\n" Dec 29 06:33:56.551: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [httpd] Dec 29 06:33:56.551: INFO: Waiting up to 5m0s for pod "httpd" in namespace "kubectl-2575" to be "running and ready" Dec 29 06:33:56.581: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 30.170361ms Dec 29 06:33:56.581: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:33:58.612: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2.061702783s Dec 29 06:33:58.612: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:00.612: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4.061224954s Dec 29 06:34:00.612: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:02.611: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 6.060579214s Dec 29 06:34:02.611: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:04.633: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 8.08192715s Dec 29 06:34:04.633: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:06.611: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 10.060453594s Dec 29 06:34:06.611: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:08.614: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 12.063584803s Dec 29 06:34:08.614: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:10.611: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 14.060332332s Dec 29 06:34:10.611: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:12.611: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 16.060802457s Dec 29 06:34:12.611: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:14.613: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 18.062371749s Dec 29 06:34:14.613: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:16.616: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 20.065250541s Dec 29 06:34:16.616: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-0b6ab0d2cddb81acd' to be 'Running' but was 'Pending' Dec 29 06:34:18.614: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 22.063068874s Dec 29 06:34:18.614: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-0b6ab0d2cddb81acd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC }] Dec 29 06:34:20.612: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 24.061256654s Dec 29 06:34:20.612: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-0b6ab0d2cddb81acd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC }] Dec 29 06:34:22.611: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 26.060373847s Dec 29 06:34:22.611: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-0b6ab0d2cddb81acd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC }] Dec 29 06:34:24.611: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 28.060845581s Dec 29 06:34:24.612: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-0b6ab0d2cddb81acd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC }] Dec 29 06:34:26.611: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 30.060391553s Dec 29 06:34:26.611: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-0b6ab0d2cddb81acd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:56 +0000 UTC }] Dec 29 06:34:28.612: INFO: Pod "httpd": Phase="Running", Reason="", readiness=true. Elapsed: 32.061537928s Dec 29 06:34:28.612: INFO: Pod "httpd" satisfied condition "running and ready" Dec 29 06:34:28.612: INFO: Wanted all 1 pods to be running and ready. Result: true. Pods: [httpd] [It] should contain last line of the log test/e2e/kubectl/kubectl.go:653 �[1mSTEP:�[0m executing a command with run �[38;5;243m12/29/22 06:34:28.612�[0m Dec 29 06:34:28.612: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2575 run run-log-test --image=registry.k8s.io/e2e-test-images/busybox:1.29-4 --restart=OnFailure --pod-running-timeout=2m0s -- sh -c sleep 10; seq 100 | while read i; do echo $i; sleep 0.01; done; echo EOF' Dec 29 06:34:28.830: INFO: stderr: "" Dec 29 06:34:28.830: INFO: stdout: "pod/run-log-test created\n" Dec 29 06:34:28.830: INFO: Waiting up to 5m0s for 1 pods to be running and ready, or succeeded: [run-log-test] Dec 29 06:34:28.830: INFO: Waiting up to 5m0s for pod "run-log-test" in namespace "kubectl-2575" to be "running and ready, or succeeded" Dec 29 06:34:28.860: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 30.349344ms Dec 29 06:34:28.860: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:30.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2.061422654s Dec 29 06:34:30.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:32.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4.06131127s Dec 29 06:34:32.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:34.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 6.061208037s Dec 29 06:34:34.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:36.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 8.061645685s Dec 29 06:34:36.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:38.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 10.061899873s Dec 29 06:34:38.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:40.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 12.060757488s Dec 29 06:34:40.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:42.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 14.061643533s Dec 29 06:34:42.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:44.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 16.061414681s Dec 29 06:34:44.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:46.893: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 18.063253293s Dec 29 06:34:46.893: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:48.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 20.061055151s Dec 29 06:34:48.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:50.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 22.061713634s Dec 29 06:34:50.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:52.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 24.061645116s Dec 29 06:34:52.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:54.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 26.061441849s Dec 29 06:34:54.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:56.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 28.061418589s Dec 29 06:34:56.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:58.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 30.061577914s Dec 29 06:34:58.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:00.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 32.061361385s Dec 29 06:35:00.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:02.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 34.061644359s Dec 29 06:35:02.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:04.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 36.061542523s Dec 29 06:35:04.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:06.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 38.061848411s Dec 29 06:35:06.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:08.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 40.061228337s Dec 29 06:35:08.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:10.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 42.06125314s Dec 29 06:35:10.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:12.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 44.061294012s Dec 29 06:35:12.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:14.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 46.061488706s Dec 29 06:35:14.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:16.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 48.061631745s Dec 29 06:35:16.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:18.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 50.062156003s Dec 29 06:35:18.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:20.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 52.061320068s Dec 29 06:35:20.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:22.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 54.06125263s Dec 29 06:35:22.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:24.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 56.061282953s Dec 29 06:35:24.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:26.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 58.061151989s Dec 29 06:35:26.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:28.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.061207713s Dec 29 06:35:28.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:30.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.06129465s Dec 29 06:35:30.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:32.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.061205401s Dec 29 06:35:32.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:34.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.062573281s Dec 29 06:35:34.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:36.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.061071317s Dec 29 06:35:36.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:38.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.061634413s Dec 29 06:35:38.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:40.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.060742818s Dec 29 06:35:40.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:42.894: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.064129918s Dec 29 06:35:42.894: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:44.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.061705467s Dec 29 06:35:44.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:46.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.061150454s Dec 29 06:35:46.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:48.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.06058766s Dec 29 06:35:48.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:50.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.061168813s Dec 29 06:35:50.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:52.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.061703204s Dec 29 06:35:52.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:54.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.061612998s Dec 29 06:35:54.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:56.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.061364392s Dec 29 06:35:56.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:58.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.06161578s Dec 29 06:35:58.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:00.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.062024532s Dec 29 06:36:00.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:02.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.061205285s Dec 29 06:36:02.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:04.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.061325827s Dec 29 06:36:04.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:06.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.060918734s Dec 29 06:36:06.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:08.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.060758482s Dec 29 06:36:08.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:10.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.061064228s Dec 29 06:36:10.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:12.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.0612175s Dec 29 06:36:12.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:14.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.061525953s Dec 29 06:36:14.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:16.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.060981132s Dec 29 06:36:16.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:18.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.061296745s Dec 29 06:36:18.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:20.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.061686824s Dec 29 06:36:20.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:22.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.06120257s Dec 29 06:36:22.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:24.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.061621685s Dec 29 06:36:24.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:26.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.061030796s Dec 29 06:36:26.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:28.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.060894767s Dec 29 06:36:28.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:30.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.061390307s Dec 29 06:36:30.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:32.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.06152269s Dec 29 06:36:32.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:34.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.061580066s Dec 29 06:36:34.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:36.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.061118287s Dec 29 06:36:36.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:38.899: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.069359173s Dec 29 06:36:38.899: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:40.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.061100546s Dec 29 06:36:40.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:42.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.06106234s Dec 29 06:36:42.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:44.893: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.063407987s Dec 29 06:36:44.893: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:46.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.061211692s Dec 29 06:36:46.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:48.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.060647197s Dec 29 06:36:48.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:50.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.060514017s Dec 29 06:36:50.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:52.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.061601045s Dec 29 06:36:52.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:54.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.061050409s Dec 29 06:36:54.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:56.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.061079333s Dec 29 06:36:56.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:58.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.061252711s Dec 29 06:36:58.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:00.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.060640905s Dec 29 06:37:00.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:02.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.061308962s Dec 29 06:37:02.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:04.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.06166916s Dec 29 06:37:04.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:06.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.061002422s Dec 29 06:37:06.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:08.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.061050386s Dec 29 06:37:08.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:10.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.061331239s Dec 29 06:37:10.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:12.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.06110744s Dec 29 06:37:12.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:14.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.060933912s Dec 29 06:37:14.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:16.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.061337005s Dec 29 06:37:16.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:18.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.061598295s Dec 29 06:37:18.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:20.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.061143646s Dec 29 06:37:20.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:22.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.061264349s Dec 29 06:37:22.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:24.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.061768589s Dec 29 06:37:24.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:26.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.061826324s Dec 29 06:37:26.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:28.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.062038414s Dec 29 06:37:28.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:30.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.061447765s Dec 29 06:37:30.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:32.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.061389512s Dec 29 06:37:32.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:34.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.06133095s Dec 29 06:37:34.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:36.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.061252671s Dec 29 06:37:36.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:38.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.060948146s Dec 29 06:37:38.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:40.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.061576867s Dec 29 06:37:40.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:42.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.061189153s Dec 29 06:37:42.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:44.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.061525477s Dec 29 06:37:44.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:46.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.061403386s Dec 29 06:37:46.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:48.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.061480921s Dec 29 06:37:48.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:50.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.061749894s Dec 29 06:37:50.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:52.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.061208329s Dec 29 06:37:52.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:54.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.061592679s Dec 29 06:37:54.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:56.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.060614355s Dec 29 06:37:56.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:58.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.061407233s Dec 29 06:37:58.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:00.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.06068599s Dec 29 06:38:00.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:02.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.061318154s Dec 29 06:38:02.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:04.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.061723295s Dec 29 06:38:04.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:06.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.061166864s Dec 29 06:38:06.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:08.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.061507742s Dec 29 06:38:08.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:10.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.061275893s Dec 29 06:38:10.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:12.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.061106242s Dec 29 06:38:12.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:14.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.061634254s Dec 29 06:38:14.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:16.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.061330452s Dec 29 06:38:16.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:18.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.061522871s Dec 29 06:38:18.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:20.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.061314785s Dec 29 06:38:20.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:22.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.061409206s Dec 29 06:38:22.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:24.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.061081147s Dec 29 06:38:24.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:26.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.061402436s Dec 29 06:38:26.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:28.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.060631465s Dec 29 06:38:28.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:30.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.060736427s Dec 29 06:38:30.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:32.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.06129995s Dec 29 06:38:32.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:34.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.06127335s Dec 29 06:38:34.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:36.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.061259985s Dec 29 06:38:36.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:38.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.061286198s Dec 29 06:38:38.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:40.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.060950456s Dec 29 06:38:40.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:42.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.061208219s Dec 29 06:38:42.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:44.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.061569076s Dec 29 06:38:44.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:46.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.061232782s Dec 29 06:38:46.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:48.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.06164522s Dec 29 06:38:48.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:50.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.06094724s Dec 29 06:38:50.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:52.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.060796626s Dec 29 06:38:52.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:54.896: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.065746014s Dec 29 06:38:54.896: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:56.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.060787179s Dec 29 06:38:56.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:58.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.061498445s Dec 29 06:38:58.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:00.893: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.063315044s Dec 29 06:39:00.893: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:02.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.061136181s Dec 29 06:39:02.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:04.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.062065095s Dec 29 06:39:04.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:06.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.061115579s Dec 29 06:39:06.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:08.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.060617916s Dec 29 06:39:08.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:10.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.061183395s Dec 29 06:39:10.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:12.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.061538746s Dec 29 06:39:12.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:14.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.062292328s Dec 29 06:39:14.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:16.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.062094245s Dec 29 06:39:16.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:18.893: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.062804931s Dec 29 06:39:18.893: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:20.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.060617359s Dec 29 06:39:20.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:22.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.06184886s Dec 29 06:39:22.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:24.891: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.061305925s Dec 29 06:39:24.891: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:26.890: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.060646554s Dec 29 06:39:26.890: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:28.892: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.062093588s Dec 29 06:39:28.892: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:28.922: INFO: Pod "run-log-test": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.09207715s Dec 29 06:39:28.922: INFO: Error evaluating pod condition running and ready, or succeeded: want pod 'run-log-test' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:28.922: INFO: Pod run-log-test failed to be running and ready, or succeeded. Dec 29 06:39:28.922: INFO: Wanted all 1 pods to be running and ready, or succeeded. Result: false. Pods: [run-log-test] Dec 29 06:39:28.922: FAIL: Pod for run-log-test was not ready Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.9() test/e2e/kubectl/kubectl.go:660 +0x265 [AfterEach] Simple pod test/e2e/kubectl/kubectl.go:417 �[1mSTEP:�[0m using delete to clean up resources �[38;5;243m12/29/22 06:39:28.922�[0m Dec 29 06:39:28.922: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2575 delete --grace-period=0 --force -f -' Dec 29 06:39:29.169: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Dec 29 06:39:29.169: INFO: stdout: "pod \"httpd\" force deleted\n" Dec 29 06:39:29.169: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2575 get rc,svc -l name=httpd --no-headers' Dec 29 06:39:29.381: INFO: stderr: "No resources found in kubectl-2575 namespace.\n" Dec 29 06:39:29.381: INFO: stdout: "" Dec 29 06:39:29.381: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2575 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Dec 29 06:39:29.578: INFO: stderr: "" Dec 29 06:39:29.578: INFO: stdout: "" [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Dec 29 06:39:29.578: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:39:29.61�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-2575". �[38;5;243m12/29/22 06:39:29.61�[0m �[1mSTEP:�[0m Found 17 events. �[38;5;243m12/29/22 06:39:29.64�[0m Dec 29 06:39:29.640: INFO: At 2022-12-29 06:33:56 +0000 UTC - event for httpd: {default-scheduler } Scheduled: Successfully assigned kubectl-2575/httpd to i-0b6ab0d2cddb81acd Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:00 +0000 UTC - event for httpd: {kubelet i-0b6ab0d2cddb81acd} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:00 +0000 UTC - event for httpd: {kubelet i-0b6ab0d2cddb81acd} Created: Created container httpd Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:00 +0000 UTC - event for httpd: {kubelet i-0b6ab0d2cddb81acd} Started: Started container httpd Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:28 +0000 UTC - event for run-log-test: {default-scheduler } Scheduled: Successfully assigned kubectl-2575/run-log-test to i-07d5c844ec1aba791 Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:30 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "83f1e6d107aa7a3b4403ccf187070a941542124087649cad18b6acf7cc760df2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:43 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d3f5e034597855aebf726f0159c3c6c21dc2da40521e19c6ec58eb8425a8b4db": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:34:54 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "29da08a3ebdc595487add4d3bc09725392a0b2547ce52a65bae365fd742e3ed0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:35:05 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4303d7f402088398a6ba4319f74a60c6422eb361b8376b9b5938d22ea21831d6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:35:16 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f7da8c38de16eecf0bdc03174044909068ba7aaa20a2ddc215f8c8676a715cba": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:35:28 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "12a065f072487d41d83ece07aa9ad94ff1fdbc40a527f1a9aee66438126d7040": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:35:42 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a57177cf306f4a8ddc5c38a1dc377d8b03373e85e8096313d92e87f239bd674e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:35:53 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "500cb3d484838df50ec178a1dc3868fe4e2669644996531631ac16b404636355": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:36:07 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0e68ace692d79b25443c7481fe1c0124f40ba5d307bb8b7a65c7bd522ffc9d5c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:36:18 +0000 UTC - event for run-log-test: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3a1a04f34d25b6981a8e7825bc12689c4a95be833399566e933a0e45e2a0abd4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:29.640: INFO: At 2022-12-29 06:39:29 +0000 UTC - event for httpd: {kubelet i-0b6ab0d2cddb81acd} Unhealthy: Readiness probe failed: Get "http://172.20.32.34:80/": dial tcp 172.20.32.34:80: connect: connection refused Dec 29 06:39:29.640: INFO: At 2022-12-29 06:39:29 +0000 UTC - event for httpd: {kubelet i-0b6ab0d2cddb81acd} Killing: Stopping container httpd Dec 29 06:39:29.670: INFO: POD NODE PHASE GRACE CONDITIONS Dec 29 06:39:29.671: INFO: run-log-test i-07d5c844ec1aba791 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:28 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:28 +0000 UTC ContainersNotReady containers with unready status: [run-log-test]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:28 +0000 UTC ContainersNotReady containers with unready status: [run-log-test]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:34:28 +0000 UTC }] Dec 29 06:39:29.671: INFO: Dec 29 06:39:29.702: INFO: Unable to fetch kubectl-2575/run-log-test/run-log-test logs: the server rejected our request for an unknown reason (get pods run-log-test) Dec 29 06:39:29.733: INFO: Logging node info for node i-0345e6864820ace32 Dec 29 06:39:29.764: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 18566 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:36:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:36:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-02311d0446177783e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02311d0446177783e,DevicePath:,},},Config:nil,},} Dec 29 06:39:29.764: INFO: Logging kubelet events for node i-0345e6864820ace32 Dec 29 06:39:29.797: INFO: Logging pods the kubelet thinks is on node i-0345e6864820ace32 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gtk84 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-ck56n started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-mcnp2 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: pod-subpath-test-dynamicpv-lx9g started at 2022-12-29 06:36:41 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Init container init-volume-dynamicpv-lx9g ready: false, restart count 0 Dec 29 06:39:29.836: INFO: Container test-container-subpath-dynamicpv-lx9g ready: false, restart count 0 Dec 29 06:39:29.836: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:29.836: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:29.836: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:29.836: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:29.836: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:29.836: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: pod-315df80e-0f06-4d38-875a-df43417f95d3 started at 2022-12-29 06:37:23 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container test-container ready: false, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dwftm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvtcq started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:29.836: INFO: affinity-nodeport-timeout-d4q9q started at 2022-12-29 06:34:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Dec 29 06:39:29.836: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container coredns ready: true, restart count 0 Dec 29 06:39:29.836: INFO: pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 started at 2022-12-29 06:37:26 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container nginx ready: false, restart count 0 Dec 29 06:39:29.836: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:29.836: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.024: INFO: Latency metrics for node i-0345e6864820ace32 Dec 29 06:39:30.024: INFO: Logging node info for node i-06be1dd8c97acf782 Dec 29 06:39:30.054: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 18030 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:36:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:39:30.055: INFO: Logging kubelet events for node i-06be1dd8c97acf782 Dec 29 06:39:30.087: INFO: Logging pods the kubelet thinks is on node i-06be1dd8c97acf782 Dec 29 06:39:30.130: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container kube-controller-manager ready: true, restart count 2 Dec 29 06:39:30.130: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:30.130: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:30.130: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:30.130: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container kops-controller ready: true, restart count 0 Dec 29 06:39:30.130: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded) Dec 29 06:39:30.130: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:30.130: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:39:30.130: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded) Dec 29 06:39:30.130: INFO: Container healthcheck ready: true, restart count 0 Dec 29 06:39:30.130: INFO: Container kube-apiserver ready: true, restart count 1 Dec 29 06:39:30.130: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container dns-controller ready: true, restart count 0 Dec 29 06:39:30.130: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container cilium-operator ready: true, restart count 0 Dec 29 06:39:30.130: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Dec 29 06:39:30.130: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:39:30.130: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.130: INFO: Container kube-scheduler ready: true, restart count 0 Dec 29 06:39:30.305: INFO: Latency metrics for node i-06be1dd8c97acf782 Dec 29 06:39:30.305: INFO: Logging node info for node i-07228a57ea47d842b Dec 29 06:39:30.336: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 19447 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8000":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:39:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:39:30.336: INFO: Logging kubelet events for node i-07228a57ea47d842b Dec 29 06:39:30.372: INFO: Logging pods the kubelet thinks is on node i-07228a57ea47d842b Dec 29 06:39:30.410: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container coredns ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b9h6m started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4xmqc started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: csi-mockplugin-0 started at 2022-12-29 06:33:33 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:30.410: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:39:30.410: INFO: Container driver-registrar ready: true, restart count 0 Dec 29 06:39:30.410: INFO: Container mock ready: true, restart count 0 Dec 29 06:39:30.410: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:30.410: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:30.410: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:30.410: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container autoscaler ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4l6sw started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: hostexec-i-07228a57ea47d842b-9fc7b started at 2022-12-29 06:39:25 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-57b2m started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:30.410: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b6wc8 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.410: INFO: pod-592477cc-870d-47db-b10f-254f00e2207c started at 2022-12-29 06:39:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:30.410: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.410: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.604: INFO: Latency metrics for node i-07228a57ea47d842b Dec 29 06:39:30.605: INFO: Logging node info for node i-07d5c844ec1aba791 Dec 29 06:39:30.636: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 17496 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6856":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:34:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2022-12-29 06:34:35 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:34:28 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:34:28 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:34:28 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:34:28 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf,DevicePath:,},},Config:nil,},} Dec 29 06:39:30.636: INFO: Logging kubelet events for node i-07d5c844ec1aba791 Dec 29 06:39:30.678: INFO: Logging pods the kubelet thinks is on node i-07d5c844ec1aba791 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: hostexec-i-07d5c844ec1aba791-9pkkk started at 2022-12-29 06:34:29 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:30.719: INFO: test-pod-2 started at 2022-12-29 06:34:22 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container token-test ready: false, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-j7svz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: run-log-test started at 2022-12-29 06:34:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container run-log-test ready: false, restart count 0 Dec 29 06:39:30.719: INFO: pod-0 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container donothing ready: true, restart count 0 Dec 29 06:39:30.719: INFO: test-pod-1 started at 2022-12-29 06:34:22 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container token-test ready: false, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zlzqg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:30.719: INFO: affinity-nodeport-timeout-cxhcj started at 2022-12-29 06:34:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Dec 29 06:39:30.719: INFO: local-injector started at 2022-12-29 06:34:40 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container local-injector ready: false, restart count 0 Dec 29 06:39:30.719: INFO: pod-1 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container donothing ready: false, restart count 0 Dec 29 06:39:30.719: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:30.719: INFO: Container jessie-querier ready: false, restart count 0 Dec 29 06:39:30.719: INFO: Container querier ready: false, restart count 0 Dec 29 06:39:30.719: INFO: Container webserver ready: false, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9v6vz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:30.719: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:30.719: INFO: externalname-service-rffwc started at 2022-12-29 06:34:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container externalname-service ready: false, restart count 0 Dec 29 06:39:30.719: INFO: httpd started at 2022-12-29 06:34:47 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container httpd ready: false, restart count 0 Dec 29 06:39:30.719: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: pod-73af6e22-e126-427f-bec8-33101a9f6c91 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:30.719: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:30.719: INFO: pod-51d122ab-7cb4-4cb1-bd00-01572790928b started at 2022-12-29 06:34:32 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container test-container ready: false, restart count 0 Dec 29 06:39:30.719: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:33:49 +0000 UTC (0+7 container statuses recorded) Dec 29 06:39:30.719: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container csi-snapshotter ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container hostpath ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:30.719: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:30.719: INFO: test-pod-3 started at 2022-12-29 06:34:22 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:30.719: INFO: Container token-test ready: true, restart count 0 Dec 29 06:39:30.980: INFO: Latency metrics for node i-07d5c844ec1aba791 Dec 29 06:39:30.980: INFO: Logging node info for node i-0b6ab0d2cddb81acd Dec 29 06:39:31.016: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 19694 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:33:36 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:39:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:13 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:13 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:13 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:13 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc,DevicePath:,},},Config:nil,},} Dec 29 06:39:31.017: INFO: Logging kubelet events for node i-0b6ab0d2cddb81acd Dec 29 06:39:31.052: INFO: Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd Dec 29 06:39:31.093: INFO: hostexec-i-0b6ab0d2cddb81acd-rr86n started at 2022-12-29 06:34:09 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:31.093: INFO: hostexec-i-0b6ab0d2cddb81acd-h4l56 started at 2022-12-29 06:33:38 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:31.093: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:31.093: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:31.093: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:31.093: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:31.093: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d8lgx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-8qfzg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:31.093: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container csi-resizer ready: false, restart count 0 Dec 29 06:39:31.093: INFO: local-injector started at 2022-12-29 06:34:41 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container local-injector ready: true, restart count 0 Dec 29 06:39:31.093: INFO: externalname-service-56kkw started at 2022-12-29 06:34:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container externalname-service ready: false, restart count 0 Dec 29 06:39:31.093: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:31.093: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:39:31.093: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:39:31.093: INFO: Container mock ready: false, restart count 0 Dec 29 06:39:31.093: INFO: affinity-nodeport-timeout-jg77b started at 2022-12-29 06:34:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container affinity-nodeport-timeout ready: false, restart count 0 Dec 29 06:39:31.093: INFO: local-client started at 2022-12-29 06:34:36 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container local-client ready: false, restart count 0 Dec 29 06:39:31.093: INFO: local-injector started at 2022-12-29 06:34:40 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container local-injector ready: false, restart count 0 Dec 29 06:39:31.093: INFO: pod-subpath-test-preprovisionedpv-kj4k started at 2022-12-29 06:34:41 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Init container init-volume-preprovisionedpv-kj4k ready: false, restart count 0 Dec 29 06:39:31.093: INFO: Container test-container-subpath-preprovisionedpv-kj4k ready: false, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6h6vh started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 started at 2022-12-29 06:33:56 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:31.093: INFO: hostexec-i-0b6ab0d2cddb81acd-qppr8 started at 2022-12-29 06:34:30 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:31.093: INFO: pod-fcc33ef1-949e-4705-aee2-05c8a8c6c3a6 started at 2022-12-29 06:34:21 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: hostexec-i-0b6ab0d2cddb81acd-th8nk started at 2022-12-29 06:34:22 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:31.093: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container donothing ready: true, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.093: INFO: pod-subpath-test-dynamicpv-t84v started at 2022-12-29 06:39:29 +0000 UTC (0+2 container statuses recorded) Dec 29 06:39:31.093: INFO: Container test-container-subpath-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:39:31.093: INFO: Container test-container-volume-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:39:31.093: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:31.093: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:31.269: INFO: Latency metrics for node i-0b6ab0d2cddb81acd [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-2575" for this suite. �[38;5;243m12/29/22 06:39:31.269�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\ssupport\sexec\sthrough\skubectl\sproxy$'
test/e2e/kubectl/kubectl.go:415 k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.1() test/e2e/kubectl/kubectl.go:415 +0x245from junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:34:46.037�[0m Dec 29 06:34:46.037: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m12/29/22 06:34:46.038�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:34:46.129�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:34:46.188�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [BeforeEach] Simple pod test/e2e/kubectl/kubectl.go:411 �[1mSTEP:�[0m creating the pod from �[38;5;243m12/29/22 06:34:46.247�[0m Dec 29 06:34:46.247: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2621 create -f -' Dec 29 06:34:47.405: INFO: stderr: "" Dec 29 06:34:47.405: INFO: stdout: "pod/httpd created\n" Dec 29 06:34:47.405: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [httpd] Dec 29 06:34:47.406: INFO: Waiting up to 5m0s for pod "httpd" in namespace "kubectl-2621" to be "running and ready" Dec 29 06:34:47.436: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 30.111206ms Dec 29 06:34:47.436: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:49.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2.060397486s Dec 29 06:34:49.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:51.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4.060842831s Dec 29 06:34:51.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:53.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 6.060809789s Dec 29 06:34:53.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:55.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 8.060989281s Dec 29 06:34:55.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:57.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 10.061112828s Dec 29 06:34:57.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:34:59.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 12.06151008s Dec 29 06:34:59.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:01.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 14.06043614s Dec 29 06:35:01.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:03.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 16.060878915s Dec 29 06:35:03.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:05.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 18.060996692s Dec 29 06:35:05.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:07.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 20.060814649s Dec 29 06:35:07.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:09.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 22.061028608s Dec 29 06:35:09.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:11.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 24.060516636s Dec 29 06:35:11.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:13.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 26.0614173s Dec 29 06:35:13.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:15.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 28.061011253s Dec 29 06:35:15.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:17.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 30.060968863s Dec 29 06:35:17.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:19.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 32.060746963s Dec 29 06:35:19.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:21.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 34.060440367s Dec 29 06:35:21.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:23.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 36.060889266s Dec 29 06:35:23.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:25.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 38.060669498s Dec 29 06:35:25.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:27.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 40.061160338s Dec 29 06:35:27.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:29.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 42.061138387s Dec 29 06:35:29.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:31.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 44.061427475s Dec 29 06:35:31.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:33.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 46.060806435s Dec 29 06:35:33.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:35.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 48.060670534s Dec 29 06:35:35.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:37.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 50.060812925s Dec 29 06:35:37.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:39.471: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 52.065040682s Dec 29 06:35:39.471: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:41.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 54.06040852s Dec 29 06:35:41.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:43.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 56.060784736s Dec 29 06:35:43.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:45.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 58.061067424s Dec 29 06:35:45.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:47.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.060780636s Dec 29 06:35:47.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:49.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.061729062s Dec 29 06:35:49.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:51.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.061643336s Dec 29 06:35:51.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:53.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.061236609s Dec 29 06:35:53.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:55.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.061513127s Dec 29 06:35:55.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:57.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.060990341s Dec 29 06:35:57.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:35:59.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.06132103s Dec 29 06:35:59.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:01.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.060834063s Dec 29 06:36:01.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:03.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.060830672s Dec 29 06:36:03.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:05.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.061166851s Dec 29 06:36:05.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:07.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.061566404s Dec 29 06:36:07.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:09.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.061642766s Dec 29 06:36:09.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:11.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.06074903s Dec 29 06:36:11.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:13.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.061066294s Dec 29 06:36:13.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:15.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.061419701s Dec 29 06:36:15.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:17.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.061425589s Dec 29 06:36:17.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:19.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.061113886s Dec 29 06:36:19.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:21.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.060731537s Dec 29 06:36:21.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:23.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.060780547s Dec 29 06:36:23.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:25.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.060705448s Dec 29 06:36:25.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:27.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.060681337s Dec 29 06:36:27.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:29.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.060938919s Dec 29 06:36:29.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:31.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.060671821s Dec 29 06:36:31.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:33.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.060797013s Dec 29 06:36:33.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:35.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.060745589s Dec 29 06:36:35.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:37.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.060977359s Dec 29 06:36:37.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:39.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.061203685s Dec 29 06:36:39.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:41.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.06133817s Dec 29 06:36:41.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:43.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.060435645s Dec 29 06:36:43.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:45.469: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.06312327s Dec 29 06:36:45.469: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:47.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.061242686s Dec 29 06:36:47.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:49.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.06057268s Dec 29 06:36:49.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:51.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.060537335s Dec 29 06:36:51.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:53.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.061467316s Dec 29 06:36:53.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:55.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.060819249s Dec 29 06:36:55.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:57.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.060575125s Dec 29 06:36:57.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:36:59.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.061277231s Dec 29 06:36:59.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:01.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.061237819s Dec 29 06:37:01.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:03.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.060996661s Dec 29 06:37:03.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:05.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.061136449s Dec 29 06:37:05.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:07.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.06106507s Dec 29 06:37:07.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:09.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.061261641s Dec 29 06:37:09.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:11.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.060523503s Dec 29 06:37:11.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:13.468: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.062966458s Dec 29 06:37:13.469: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:15.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.061890237s Dec 29 06:37:15.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:17.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.061390853s Dec 29 06:37:17.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:19.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.060183242s Dec 29 06:37:19.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:21.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.060808574s Dec 29 06:37:21.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:23.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.061396183s Dec 29 06:37:23.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:25.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.060778233s Dec 29 06:37:25.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:27.468: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.062606521s Dec 29 06:37:27.468: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:29.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.061573995s Dec 29 06:37:29.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:31.471: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.065812511s Dec 29 06:37:31.471: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:33.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.060947297s Dec 29 06:37:33.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:35.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.061033618s Dec 29 06:37:35.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:37.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.060685254s Dec 29 06:37:37.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:39.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.061152426s Dec 29 06:37:39.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:41.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.060660103s Dec 29 06:37:41.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:43.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.060976974s Dec 29 06:37:43.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:45.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.060395062s Dec 29 06:37:45.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:47.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.061422098s Dec 29 06:37:47.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:49.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.060790126s Dec 29 06:37:49.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:51.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.060260538s Dec 29 06:37:51.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:53.468: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.062907696s Dec 29 06:37:53.468: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:55.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.0613016s Dec 29 06:37:55.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:57.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.061231306s Dec 29 06:37:57.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:37:59.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.061553097s Dec 29 06:37:59.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:01.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.060942373s Dec 29 06:38:01.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:03.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.06131606s Dec 29 06:38:03.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:05.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.061453215s Dec 29 06:38:05.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:07.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.061266463s Dec 29 06:38:07.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:09.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.060577253s Dec 29 06:38:09.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:11.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.060560555s Dec 29 06:38:11.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:13.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.061182706s Dec 29 06:38:13.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:15.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.061201589s Dec 29 06:38:15.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:17.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.061364663s Dec 29 06:38:17.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:19.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.060938778s Dec 29 06:38:19.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:21.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.060818419s Dec 29 06:38:21.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:23.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.060582058s Dec 29 06:38:23.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:25.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.060732499s Dec 29 06:38:25.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:27.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.061044174s Dec 29 06:38:27.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:29.468: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.062288465s Dec 29 06:38:29.468: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:31.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.060936349s Dec 29 06:38:31.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:33.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.061023656s Dec 29 06:38:33.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:35.469: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.063255091s Dec 29 06:38:35.469: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:37.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.060714592s Dec 29 06:38:37.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:39.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.060970131s Dec 29 06:38:39.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:41.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.061099827s Dec 29 06:38:41.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:43.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.06109948s Dec 29 06:38:43.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:45.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.061024357s Dec 29 06:38:45.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:47.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.060904066s Dec 29 06:38:47.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:49.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.061110337s Dec 29 06:38:49.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:51.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.061085166s Dec 29 06:38:51.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:53.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.061001862s Dec 29 06:38:53.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:55.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.061526617s Dec 29 06:38:55.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:57.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.060862425s Dec 29 06:38:57.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:38:59.473: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.067238393s Dec 29 06:38:59.473: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:01.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.061151683s Dec 29 06:39:01.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:03.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.061006151s Dec 29 06:39:03.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:05.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.06026898s Dec 29 06:39:05.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:07.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.060715322s Dec 29 06:39:07.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:09.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.061452299s Dec 29 06:39:09.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:11.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.060744506s Dec 29 06:39:11.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:13.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.061609829s Dec 29 06:39:13.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:15.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.061156781s Dec 29 06:39:15.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:17.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.06095715s Dec 29 06:39:17.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:19.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.060806492s Dec 29 06:39:19.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:21.474: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.068005075s Dec 29 06:39:21.474: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:23.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.060186308s Dec 29 06:39:23.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:25.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.060126246s Dec 29 06:39:25.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:27.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.060500619s Dec 29 06:39:27.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:29.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.061106622s Dec 29 06:39:29.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:31.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.061351973s Dec 29 06:39:31.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:33.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.060247347s Dec 29 06:39:33.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:35.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.060160375s Dec 29 06:39:35.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:37.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.060812005s Dec 29 06:39:37.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:39.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.060746838s Dec 29 06:39:39.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:41.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.060763311s Dec 29 06:39:41.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:43.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.061289412s Dec 29 06:39:43.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:45.466: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.060544228s Dec 29 06:39:45.466: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:47.467: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.06113665s Dec 29 06:39:47.467: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:47.497: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.09105286s Dec 29 06:39:47.497: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending' Dec 29 06:39:47.497: INFO: Pod httpd failed to be running and ready. Dec 29 06:39:47.497: INFO: Wanted all 1 pods to be running and ready. Result: false. Pods: [httpd] Dec 29 06:39:47.497: FAIL: Expected <bool>: false to equal <bool>: true Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.1() test/e2e/kubectl/kubectl.go:415 +0x245 [AfterEach] Simple pod test/e2e/kubectl/kubectl.go:417 �[1mSTEP:�[0m using delete to clean up resources �[38;5;243m12/29/22 06:39:47.497�[0m Dec 29 06:39:47.497: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2621 delete --grace-period=0 --force -f -' Dec 29 06:39:47.718: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Dec 29 06:39:47.718: INFO: stdout: "pod \"httpd\" force deleted\n" Dec 29 06:39:47.718: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2621 get rc,svc -l name=httpd --no-headers' Dec 29 06:39:47.932: INFO: stderr: "No resources found in kubectl-2621 namespace.\n" Dec 29 06:39:47.932: INFO: stdout: "" Dec 29 06:39:47.933: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2621 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Dec 29 06:39:48.117: INFO: stderr: "" Dec 29 06:39:48.117: INFO: stdout: "" [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Dec 29 06:39:48.117: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m12/29/22 06:39:48.149�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-2621". �[38;5;243m12/29/22 06:39:48.149�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m12/29/22 06:39:48.18�[0m Dec 29 06:39:48.180: INFO: At 2022-12-29 06:34:47 +0000 UTC - event for httpd: {default-scheduler } Scheduled: Successfully assigned kubectl-2621/httpd to i-07d5c844ec1aba791 Dec 29 06:39:48.180: INFO: At 2022-12-29 06:34:47 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b2cda6bd2a1aa557768b2a3a021f5667ba72ea0181c441f46a5edccb50f10cfb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:35:02 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6017e6aee397f380faa9a21bc84a6613cb25658aec839ffc9ece018bc67f64d0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:35:14 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6b0e167c9d39f3790a15496238994080c10387b96e0dc7f655bec7c1cdb630e0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:35:26 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b9f5bf24826db5dcce24049a704e9c62147ca5c78704f94da92bee497685ac65": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:35:38 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5c64ba27eb6f3e636f98de485a4fcdce787c7c64c9a28f29a101c323d15fbd7c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:35:49 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d4a07f5d52393442037af263bce3dff16ceacdf9415913a05e3e565797959f6a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:36:02 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "772406419ce87ecfcc1eaf8e7cd6aca4acd9de6219861981dece1c9d0aabcd40": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:36:15 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "abd9b58d3b3fd9504f1b924f929fa011792cb81618546dac5c3e89a682ee372e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:36:27 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0f5520e1d324e21b0dac1f969dce171b6f94c713830398932ded223bd4c9de83": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.180: INFO: At 2022-12-29 06:36:42 +0000 UTC - event for httpd: {kubelet i-07d5c844ec1aba791} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3e655402d0dbd55f6509058b2a5d134aa93a1a0c059e8acc62be3eacf3888298": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Dec 29 06:39:48.210: INFO: POD NODE PHASE GRACE CONDITIONS Dec 29 06:39:48.210: INFO: Dec 29 06:39:48.241: INFO: Logging node info for node i-0345e6864820ace32 Dec 29 06:39:48.271: INFO: Node Info: &Node{ObjectMeta:{i-0345e6864820ace32 d24e217a-7498-44e8-ad0a-bde5d9d75ec2 18566 0 2022-12-29 06:25:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0345e6864820ace32 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0345e6864820ace32 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.74 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0345e6864820ace32"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:36:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:36:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0345e6864820ace32,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:25:50 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:36:45 +0000 UTC,LastTransitionTime:2022-12-29 06:26:03 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.74,},NodeAddress{Type:ExternalIP,Address:52.15.211.117,},NodeAddress{Type:InternalDNS,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0345e6864820ace32.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-15-211-117.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2fde6815ed8eaf06d9808d0d5fd9fd,SystemUUID:ec2fde68-15ed-8eaf-06d9-808d0d5fd9fd,BootID:c3ceeff7-72e6-488d-a15e-963f4e62f431,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-02311d0446177783e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02311d0446177783e,DevicePath:,},},Config:nil,},} Dec 29 06:39:48.272: INFO: Logging kubelet events for node i-0345e6864820ace32 Dec 29 06:39:48.304: INFO: Logging pods the kubelet thinks is on node i-0345e6864820ace32 Dec 29 06:39:48.340: INFO: coredns-559769c974-v86nc started at 2022-12-29 06:26:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container coredns ready: true, restart count 0 Dec 29 06:39:48.340: INFO: pod3d38b6a2-3d54-46d2-bcc6-44ccdf49f6a8 started at 2022-12-29 06:37:26 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container nginx ready: false, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-sjdz4 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gtk84 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-wfqkn started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-pcfgp started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-ck56n started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-bfz2k started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-mcnp2 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: pod-subpath-test-dynamicpv-lx9g started at 2022-12-29 06:36:41 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Init container init-volume-dynamicpv-lx9g ready: false, restart count 0 Dec 29 06:39:48.340: INFO: Container test-container-subpath-dynamicpv-lx9g ready: false, restart count 0 Dec 29 06:39:48.340: INFO: cilium-w8v52 started at 2022-12-29 06:25:51 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:48.340: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:48.340: INFO: netserver-0 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container webserver ready: false, restart count 0 Dec 29 06:39:48.340: INFO: ebs-csi-node-vghzd started at 2022-12-29 06:25:51 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:48.340: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:48.340: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:48.340: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9vx2r started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: pod-315df80e-0f06-4d38-875a-df43417f95d3 started at 2022-12-29 06:37:23 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container test-container ready: false, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dwftm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvtcq started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.340: INFO: affinity-nodeport-timeout-d4q9q started at 2022-12-29 06:34:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.340: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Dec 29 06:39:48.520: INFO: Latency metrics for node i-0345e6864820ace32 Dec 29 06:39:48.520: INFO: Logging node info for node i-06be1dd8c97acf782 Dec 29 06:39:48.551: INFO: Node Info: &Node{ObjectMeta:{i-06be1dd8c97acf782 f4bca287-3671-48aa-9e15-93047571360e 18030 0 2022-12-29 06:23:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06be1dd8c97acf782 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-east-2a topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.45.230 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06be1dd8c97acf782"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-12-29 06:23:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2022-12-29 06:24:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:24:36 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2022-12-29 06:36:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-06be1dd8c97acf782,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:23:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:36:13 +0000 UTC,LastTransitionTime:2022-12-29 06:24:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.45.230,},NodeAddress{Type:ExternalIP,Address:18.119.161.223,},NodeAddress{Type:InternalDNS,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-06be1dd8c97acf782.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-119-161-223.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b354175413e563e031cbf3484612b,SystemUUID:ec2b3541-7541-3e56-3e03-1cbf3484612b,BootID:db441d82-8c66-4137-bae7-0036808c484e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43456502,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42804917,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967336,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:39:48.551: INFO: Logging kubelet events for node i-06be1dd8c97acf782 Dec 29 06:39:48.583: INFO: Logging pods the kubelet thinks is on node i-06be1dd8c97acf782 Dec 29 06:39:48.621: INFO: cilium-operator-5dd44dc49f-7tcqz started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container cilium-operator ready: true, restart count 0 Dec 29 06:39:48.621: INFO: aws-cloud-controller-manager-kt5tl started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Dec 29 06:39:48.621: INFO: etcd-manager-main-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:39:48.621: INFO: kube-scheduler-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container kube-scheduler ready: true, restart count 0 Dec 29 06:39:48.621: INFO: dns-controller-69987775c6-548mf started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container dns-controller ready: true, restart count 0 Dec 29 06:39:48.621: INFO: cilium-tc4w5 started at 2022-12-29 06:24:25 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:48.621: INFO: ebs-csi-node-d9jxk started at 2022-12-29 06:24:26 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:48.621: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:48.621: INFO: kops-controller-9dzn7 started at 2022-12-29 06:24:27 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container kops-controller ready: true, restart count 0 Dec 29 06:39:48.621: INFO: ebs-csi-controller-89b649c94-j8vxm started at 2022-12-29 06:24:28 +0000 UTC (0+5 container statuses recorded) Dec 29 06:39:48.621: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:48.621: INFO: etcd-manager-events-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container etcd-manager ready: true, restart count 0 Dec 29 06:39:48.621: INFO: kube-apiserver-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+2 container statuses recorded) Dec 29 06:39:48.621: INFO: Container healthcheck ready: true, restart count 0 Dec 29 06:39:48.621: INFO: Container kube-apiserver ready: true, restart count 1 Dec 29 06:39:48.621: INFO: kube-controller-manager-i-06be1dd8c97acf782 started at 2022-12-29 06:23:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.621: INFO: Container kube-controller-manager ready: true, restart count 2 Dec 29 06:39:48.782: INFO: Latency metrics for node i-06be1dd8c97acf782 Dec 29 06:39:48.782: INFO: Logging node info for node i-07228a57ea47d842b Dec 29 06:39:48.812: INFO: Node Info: &Node{ObjectMeta:{i-07228a57ea47d842b 76547faa-e43c-49e6-a850-ad85a028c7c6 20142 0 2022-12-29 06:25:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07228a57ea47d842b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07228a57ea47d842b topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.93 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9651":"i-07228a57ea47d842b","ebs.csi.aws.com":"i-07228a57ea47d842b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2022-12-29 06:39:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07228a57ea47d842b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:12 +0000 UTC,LastTransitionTime:2022-12-29 06:25:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.93,},NodeAddress{Type:ExternalIP,Address:18.218.178.12,},NodeAddress{Type:InternalDNS,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07228a57ea47d842b.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-218-178-12.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2b313e320fbe8d04b7d78918a0d48f,SystemUUID:ec2b313e-320f-be8d-04b7-d78918a0d48f,BootID:1f793e0c-d279-4dd8-aae2-a89cb5e14e43,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Dec 29 06:39:48.812: INFO: Logging kubelet events for node i-07228a57ea47d842b Dec 29 06:39:48.845: INFO: Logging pods the kubelet thinks is on node i-07228a57ea47d842b Dec 29 06:39:48.884: INFO: ebs-csi-node-7q6nn started at 2022-12-29 06:25:46 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:48.884: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:48.884: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:48.884: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-xvc6g started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: pvc-volume-tester-59h2h started at 2022-12-29 06:39:45 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container volume-tester ready: false, restart count 0 Dec 29 06:39:48.884: INFO: coredns-autoscaler-7cb5c5b969-h9brm started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container autoscaler ready: true, restart count 0 Dec 29 06:39:48.884: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:38 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:48.884: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:39:48.884: INFO: Container driver-registrar ready: true, restart count 0 Dec 29 06:39:48.884: INFO: Container mock ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-s5qfj started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4l6sw started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-qnx9p started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: netserver-1 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container webserver ready: false, restart count 0 Dec 29 06:39:48.884: INFO: hostexec-i-07228a57ea47d842b-9fc7b started at 2022-12-29 06:39:25 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-57b2m started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: hostexec-i-07228a57ea47d842b-8hx2v started at 2022-12-29 06:39:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cilium-5wqfk started at 2022-12-29 06:25:46 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:48.884: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b6wc8 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: pod-592477cc-870d-47db-b10f-254f00e2207c started at 2022-12-29 06:39:28 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6ztqj started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-jcpz9 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: coredns-559769c974-zr5zz started at 2022-12-29 06:25:58 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container coredns ready: true, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-b9h6m started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:48.884: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:38 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:39:48.884: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4xmqc started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:48.884: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.092: INFO: Latency metrics for node i-07228a57ea47d842b Dec 29 06:39:49.092: INFO: Logging node info for node i-07d5c844ec1aba791 Dec 29 06:39:49.122: INFO: Node Info: &Node{ObjectMeta:{i-07d5c844ec1aba791 4a55eed5-0895-4774-aa72-319ea35ed997 20007 0 2022-12-29 06:25:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-07d5c844ec1aba791 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-07d5c844ec1aba791 topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.167 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-6856":"i-07d5c844ec1aba791","ebs.csi.aws.com":"i-07d5c844ec1aba791"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:39:34 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:39:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-07d5c844ec1aba791,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:35 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:35 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:35 +0000 UTC,LastTransitionTime:2022-12-29 06:25:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:35 +0000 UTC,LastTransitionTime:2022-12-29 06:26:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.167,},NodeAddress{Type:ExternalIP,Address:3.137.170.63,},NodeAddress{Type:InternalDNS,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-07d5c844ec1aba791.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-137-170-63.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2816c98e4e4a56d31c5da62b86b1e8,SystemUUID:ec2816c9-8e4e-4a56-d31c-5da62b86b1e8,BootID:d01bfd26-63c0-4f0d-9def-059a035d44bb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03b2cff6dadc1e21e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6856^c4a62316-8742-11ed-87f9-666e6638b7cf,DevicePath:,},},Config:nil,},} Dec 29 06:39:49.123: INFO: Logging kubelet events for node i-07d5c844ec1aba791 Dec 29 06:39:49.155: INFO: Logging pods the kubelet thinks is on node i-07d5c844ec1aba791 Dec 29 06:39:49.194: INFO: pod-73af6e22-e126-427f-bec8-33101a9f6c91 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-gdzwr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: csi-hostpathplugin-0 started at 2022-12-29 06:33:49 +0000 UTC (0+7 container statuses recorded) Dec 29 06:39:49.194: INFO: Container csi-attacher ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container csi-provisioner ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container csi-snapshotter ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container hostpath ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d6twr started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-j7svz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-k5ghk started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-g4pdf started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: pod-0 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container donothing ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-r6thv started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zlzqg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: false, restart count 0 Dec 29 06:39:49.194: INFO: affinity-nodeport-timeout-cxhcj started at 2022-12-29 06:34:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Dec 29 06:39:49.194: INFO: pod-1 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container donothing ready: false, restart count 0 Dec 29 06:39:49.194: INFO: dns-test-c172c678-9dc4-4f02-a1a7-b3fee6122f4f started at 2022-12-29 06:34:55 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:49.194: INFO: Container jessie-querier ready: false, restart count 0 Dec 29 06:39:49.194: INFO: Container querier ready: false, restart count 0 Dec 29 06:39:49.194: INFO: Container webserver ready: false, restart count 0 Dec 29 06:39:49.194: INFO: inline-volume-tester-4b2rw started at 2022-12-29 06:39:32 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container csi-volume-tester ready: false, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-rzf4t started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-9v6vz started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.194: INFO: ebs-csi-node-g2svb started at 2022-12-29 06:25:49 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:49.194: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:49.194: INFO: externalname-service-rffwc started at 2022-12-29 06:34:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container externalname-service ready: false, restart count 0 Dec 29 06:39:49.194: INFO: netserver-2 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container webserver ready: true, restart count 0 Dec 29 06:39:49.194: INFO: pod-configmaps-191e839a-8743-4828-8230-52b8b3ce9a18 started at 2022-12-29 06:39:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container agnhost-container ready: false, restart count 0 Dec 29 06:39:49.194: INFO: cilium-gxqpq started at 2022-12-29 06:25:49 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:49.194: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:49.194: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-4mhsg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.194: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.461: INFO: Latency metrics for node i-07d5c844ec1aba791 Dec 29 06:39:49.461: INFO: Logging node info for node i-0b6ab0d2cddb81acd Dec 29 06:39:49.491: INFO: Node Info: &Node{ObjectMeta:{i-0b6ab0d2cddb81acd 5773f1c1-66cd-4837-add7-b4e5e5cf262a 19970 0 2022-12-29 06:25:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-east-2 failure-domain.beta.kubernetes.io/zone:us-east-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b6ab0d2cddb81acd kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-east-2a topology.hostpath.csi/node:i-0b6ab0d2cddb81acd topology.kubernetes.io/region:us-east-2 topology.kubernetes.io/zone:us-east-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.62.9 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b6ab0d2cddb81acd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2022-12-29 06:25:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2022-12-29 06:34:04 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2022-12-29 06:39:32 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-12-29 06:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-east-2a/i-0b6ab0d2cddb81acd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-12-29 06:39:34 +0000 UTC,LastTransitionTime:2022-12-29 06:25:59 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.62.9,},NodeAddress{Type:ExternalIP,Address:3.21.158.52,},NodeAddress{Type:InternalDNS,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b6ab0d2cddb81acd.us-east-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-21-158-52.us-east-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec205c07b784a7ddb756c669b73295a8,SystemUUID:ec205c07-b784-a7dd-b756-c669b73295a8,BootID:ce3855e2-81e2-4056-bb60-3e652cc427ec,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420 kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0accc812e75c4b2dc,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08f18acb8b273c420,DevicePath:,},},Config:nil,},} Dec 29 06:39:49.492: INFO: Logging kubelet events for node i-0b6ab0d2cddb81acd Dec 29 06:39:49.527: INFO: Logging pods the kubelet thinks is on node i-0b6ab0d2cddb81acd Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-fpdhx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-zr2jm started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vgqt2 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-6h6vh started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: pod-be1e2e15-c415-4f41-a9a8-ca99a6b8b351 started at 2022-12-29 06:33:56 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container write-pod ready: false, restart count 0 Dec 29 06:39:49.567: INFO: hostexec-i-0b6ab0d2cddb81acd-qppr8 started at 2022-12-29 06:34:30 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container agnhost-container ready: true, restart count 0 Dec 29 06:39:49.567: INFO: pod-2 started at 2022-12-29 06:34:24 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container donothing ready: true, restart count 0 Dec 29 06:39:49.567: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:34 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:49.567: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:39:49.567: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:39:49.567: INFO: Container mock ready: false, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-dnfm8 started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-nvbdx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-hgh22 started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: pod-subpath-test-dynamicpv-t84v started at 2022-12-29 06:39:29 +0000 UTC (0+2 container statuses recorded) Dec 29 06:39:49.567: INFO: Container test-container-subpath-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:39:49.567: INFO: Container test-container-volume-dynamicpv-t84v ready: false, restart count 0 Dec 29 06:39:49.567: INFO: csi-mockplugin-attacher-0 started at 2022-12-29 06:39:34 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container csi-attacher ready: false, restart count 0 Dec 29 06:39:49.567: INFO: cilium-458cx started at 2022-12-29 06:25:47 +0000 UTC (1+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Init container clean-cilium-state ready: true, restart count 0 Dec 29 06:39:49.567: INFO: Container cilium-agent ready: true, restart count 0 Dec 29 06:39:49.567: INFO: ebs-csi-node-tt7f4 started at 2022-12-29 06:25:47 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:49.567: INFO: Container ebs-plugin ready: true, restart count 0 Dec 29 06:39:49.567: INFO: Container liveness-probe ready: true, restart count 0 Dec 29 06:39:49.567: INFO: Container node-driver-registrar ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-8qfzg started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: csi-mockplugin-resizer-0 started at 2022-12-29 06:39:21 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container csi-resizer ready: true, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-d8lgx started at 2022-12-29 06:34:04 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: netserver-3 started at 2022-12-29 06:39:31 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container webserver ready: false, restart count 0 Dec 29 06:39:49.567: INFO: externalname-service-56kkw started at 2022-12-29 06:34:44 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container externalname-service ready: false, restart count 0 Dec 29 06:39:49.567: INFO: local-client started at 2022-12-29 06:39:39 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container local-client ready: false, restart count 0 Dec 29 06:39:49.567: INFO: csi-mockplugin-0 started at 2022-12-29 06:39:21 +0000 UTC (0+3 container statuses recorded) Dec 29 06:39:49.567: INFO: Container csi-provisioner ready: false, restart count 0 Dec 29 06:39:49.567: INFO: Container driver-registrar ready: false, restart count 0 Dec 29 06:39:49.567: INFO: Container mock ready: false, restart count 0 Dec 29 06:39:49.567: INFO: affinity-nodeport-timeout-jg77b started at 2022-12-29 06:34:33 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container affinity-nodeport-timeout ready: false, restart count 0 Dec 29 06:39:49.567: INFO: cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c-vpnlm started at 2022-12-29 06:34:05 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container cleanup40-d07320d9-93da-4dd2-9711-0a77c95b8d0c ready: true, restart count 0 Dec 29 06:39:49.567: INFO: downwardapi-volume-cc6aea7e-266b-4611-a490-6cb3559eaf08 started at 2022-12-29 06:39:43 +0000 UTC (0+1 container statuses recorded) Dec 29 06:39:49.567: INFO: Container client-container ready: false, restart count 0 Dec 29 06:39:49.756: INFO: Latency metrics for node i-0b6ab0d2cddb81acd [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-2621" for this suite. �[38;5;243m12/29/22 06:39:49.756�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\ssupport\sinline\sexecution\sand\sattach$'
test/e2e/framework/kubectl/builder.go:87
k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.ExecOrDie({0xc001571e40?, 0x0?}, {0xc00368f500, 0xc})
test/e2e/framework/kubectl/builder.go:87 +0x1b4
k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.8()
test/e2e/kubectl/kubectl.go:637 +0xbcb
[BeforeEach] [sig-cli] Kubectl client
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m12/29/22 06:33:04.217�[0m
Dec 29 06:33:04.217: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m12/29/22 06:33:04.218�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m12/29/22 06:33:04.323�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m12/29/22 06:33:04.383�[0m
[BeforeEach] [sig-cli] Kubectl client
test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-cli] Kubectl client
test/e2e/kubectl/kubectl.go:274
[BeforeEach] Simple pod
test/e2e/kubectl/kubectl.go:411
�[1mSTEP:�[0m creating the pod from �[38;5;243m12/29/22 06:33:04.442�[0m
Dec 29 06:33:04.443: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/bf9164e7-8740-11ed-b47a-e2b55b2f9198/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-2235 create -f -'
Dec 29 06:33:05.385: INFO: stderr: ""
Dec 29 06:33:05.385: INFO: stdout: "pod/httpd created\n"
Dec 29 06:33:05.385: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [httpd]
Dec 29 06:33:05.385: INFO: Waiting up to 5m0s for pod "httpd" in namespace "kubectl-2235" to be "running and ready"
Dec 29 06:33:05.420: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 35.042869ms
Dec 29 06:33:05.420: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:07.450: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2.065809145s
Dec 29 06:33:07.450: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:09.450: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4.065007887s
Dec 29 06:33:09.450: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:11.449: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 6.064711154s
Dec 29 06:33:11.449: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:13.450: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 8.065806476s
Dec 29 06:33:13.450: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:15.449: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 10.064709424s
Dec 29 06:33:15.449: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:17.449: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 12.06485243s
Dec 29 06:33:17.450: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:19.451: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 14.06588074s
Dec 29 06:33:19.451: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:21.449: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 16.064377721s
Dec 29 06:33:21.449: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:23.449: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 18.064153392s
Dec 29 06:33:23.449: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'i-07d5c844ec1aba791' to be 'Running' but was 'Pending'
Dec 29 06:33:25.450: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 20.064992824s
Dec 29 06:33:25.450: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:27.450: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 22.065210396s
Dec 29 06:33:27.450: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:29.451: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 24.066857951s
Dec 29 06:33:29.452: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:31.450: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 26.065391542s
Dec 29 06:33:31.450: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:33.449: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 28.06430877s
Dec 29 06:33:33.449: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:35.450: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 30.065200194s
Dec 29 06:33:35.450: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:37.453: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 32.067901278s
Dec 29 06:33:37.453: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:39.450: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 34.065052987s
Dec 29 06:33:39.450: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC }]
Dec 29 06:33:41.451: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 36.065883977s
Dec 29 06:33:41.451: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'i-07d5c844ec1aba791' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-12-29 06:33:05 +0000 UTC ContainersNotReady containers with unready status: [httpd]